We urgently call for international red lines to prevent unacceptable AI risks.

Launched during the 80th session of the United Nations General Assembly, this call has broad support from prominent leaders in policy, academia, and industry.

  • 200+ prominent figures
  • 70+ organizations
  • 8 former heads of state and ministers
  • 10 Nobel laureates and recipients

Global Call for AI Red Lines

AI holds immense potential to advance human wellbeing, yet its current trajectory presents unprecedented dangers. AI could soon far surpass human capabilities and escalate risks such as engineered pandemics, widespread disinformation, large-scale manipulation of individuals including children, national and international security concerns, mass unemployment, and systematic human rights violations. 
Some advanced AI systems have already exhibited deceptive and harmful behavior, and yet these systems are being given more autonomy to take actions and make decisions in the world. Left unchecked, many experts, including those at the forefront of development, warn that it will become increasingly difficult to exert meaningful human control in the coming years. 
Governments must act decisively before the window for meaningful intervention closes. An international agreement on clear and verifiable red lines is necessary for preventing universally unacceptable risks. These red lines should build upon and enforce existing global frameworks and voluntary corporate commitments, ensuring that all advanced AI providers are accountable to shared thresholds. 
We urge governments to reach an international agreement on red lines for AI — ensuring they are operational, with robust enforcement mechanisms — by the end of 2026. 

What Signatories Say

It is in our vital common interest to prevent AI from inflicting serious and potentially irreversible damages to humanity, and we should act accordingly.
Ahmet Üzümcü

Former Director General of the Organization for the Prohibition of Chemical Weapons

For thousands of years, humans have learned—sometimes the hard way—that powerful technologies can have dangerous as well as beneficial consequences. With AI, we may not get a chance to learn from our mistakes, because AI is the first technology that can make decisions by itself, invent new ideas by itself, and escape our control. Humans must agree on clear red lines for AI before the technology reshapes society beyond our understanding and destroys the foundations of our humanity.
Yuval Noah Harari

Author of "Sapiens"

Humanity in its long history has never met intelligence higher than ours. Within a few years, we will. But we are far from being prepared for it in terms of regulations, safeguards, and governance.
Csaba Kőrösi

Former President of the UN General Assembly

The development of highly capable AI could be the most significant event in human history. It is imperative that world powers act decisively to ensure it is not the last.
Stuart Russell

Distinguished Professor of Computer Science at the University of California, Berkeley

The current race towards ever more capable and autonomous AI systems poses major risks to our societies and we urgently need international collaboration to address them. Establishing red lines is a crucial step towards preventing unacceptable AI risks.
Yoshua Bengio

2018 Turing Award Winner

This should be a major wake up call for policy makers and AI developers.
Lord Tim Clement-Jones

House of Lords' Science Innovation and Technology Spokesperson

Without AI safeguards, we may soon face epistemic chaos, engineered pandemics, and systematic human rights violation. History teaches us that when confronted with irreversible, borderless threats, cooperation is the only rational way to pursue national interests.
Maria Ressa

Nobel Peace Prize Laureate

Signatories

Joseph Stiglitz

Nobel Laureate in Economics

Professor of Finance and Business
Columbia University

Juan Manuel Santos

Former President of Colombia
Nobel Peace Prize Laureate

Chair
The Elders

Maria Ressa

Nobel Peace Prize Laureate

Co-founder and CEO
Rappler

Daron Acemoğlu

Nobel Laureate in Economics

Institute Professor
MIT

Mary Robinson

First Woman President of Ireland

Former UN High Commissioner for Human Rights

Member
The Elders

Ahmet Üzümcü

Nobel Peace Prize Recipient on Behalf of the OPCW

Former Director General
Organization for the Prohibition of Chemical Weapons (OPCW)

Senior Fellow
European Leadership Network (ELN)

Geoffrey Hinton

Nobel Laureate in Physics
Turing Award Winner

Emeritus Professor of Computer Science
University of Toronto

Yoshua Bengio

Most Cited Living Scientist
Turing Award Winner

Full Professor
Université de Montréal

Chair
International Scientific Report on the Safety of Advanced AI

Co-President and Scientific Director
LawZero

Founder and Scientific Advisor
Mila – Quebec AI Institute

John Hopfield

Nobel Laureate in Physics

Emeritus Professor
Princeton University

Yuval Noah Harari

Author of 'Sapiens'

Professor of History
Hebrew University of Jerusalem

Jennifer Doudna

Nobel Laureate in Chemistry

Professor
University of California, Berkeley
Co-developer of CRISPR-Cas9, gene-editing tool

Enrico Letta

Former Prime Minister of Italy

President
Agenzia di Ricerche e Legislazione (AREL)

Csaba Kőrösi

77th President of the UN General Assembly

Strategic Director
Blue Planet Foundation

Giorgio Parisi

Nobel Laureate in Physics

Emeritus Professor
University of Rome La Sapienza

Sir Oliver Hart

Nobel Laureate in Economics

Professor
Harvard University

Ya-Qin Zhang

Chair Professor and Dean, Institute for AI Industry Research
Tsinghua University

Former President
Baidu

Stuart Russell

Professor and Smith-Zadeh Chair in Engineering
University of California, Berkeley

Founder
Center for Human-Compatible Artificial Intelligence (CHAI)

Joseph Sifakis

Turing Award Winner

Research Director Emeritus, Verimag Lab
Université Grenoble - Alpes

Kate Crawford

TIME 100 AI

Professor
University of Southern California

Senior Principal Researcher
MSR

Wojciech Zaremba

Co-founder of OpenAI

Former Research Scientist
Facebook AI Research

Fomer Research Scientist
Google Brain

Yanis Varoufakis

Former Minister of Finance of Greece

Professor
University of Athens

Peter Norvig

Education Fellow
Stanford, Institute for Human-Center AI (HAI)

Director of Engineering
Google

George Church

Professor
Harvard Medical School & MIT

Ian Goodfellow

Principal Scientist
Google DeepMind

Inventor of generative adversarial networks

Founder of Google Brain's machine learning security research team

Andrew Chi-Chih Yao

Turing Award Winner

Professor
Tsinghua University

Sir Stephen Fry

Writer, Director, Actor

Yi Zeng

TIME 100 AI

Dean
Beijing Institute of AI Safety and Governance

Gustavo Béliz

Former Governor of Argentina

Former Minister
Government of Argentina

Former Secretary of the President
Government of Argentina

Chair
Economic and Social Council of Argentina

Baroness Beeban Kidron

Crossbench Peer
UK House of Lords

Advisor to the Institute for Ethics in AI
Oxford University

Founder and Chair
5Rights Foundation

Gbegna Sesan

Executive Director
Paradigm Initiative

Anna Ascani

Vice President
Italian Chamber of Deputies

Dan Hendrycks

TIME 100 AI

Executive Director
Center for AI Safety

Advisor
xAI

Advisor
Scale AI

Dawn Song

Professor
University of California, Berkeley

Gary Marcus

Professor Emeritus
New York University

Audrey Tang

Taiwan's Cyber Ambassador and First Digital Minister
TIME 100 AI

Senior Accelerator Fellow, Institute for Ethics in AI
University of Oxford

Maria João Rodrigues

Former Portuguese Minister of Employment

Former Member of the European Parliament 

President of the Foundation for European Progressive Studies (FEPS)

Rachel Adams

Founding CEO
Global Center on AI Governance

Adetola A.Salau

Special Adviser to the Honourable Minister of Education
Federal Ministry of Education Nigeria

Thibaut Bruttin

Director General
Reporters Without Borders (RSF)

Maria Chiara Carrozza

Former Italian Minister of Education, University and Research

Full Professor of Biomedical Engineering and Biorobotics
University of Milano-Bicocca

Daniel Kokotajlo

TIME 100 AI

Former researcher
OpenAI

Co-founder and Lead
AI Futures Project

Lord Tim Clement-Jones

Peer / Science Innovation and Technology Spokesperson
UK House of Lords

Xue Lan

TIME 100 AI

Dean, Schwarzman College
Tsinghua University

Robert Trager

Director
Oxford Martin AI Governance Initiative, University of Oxford

Marc Rotenberg

Executive Director and Founder
Center for AI and Digital Policy

Laurence Devillers

Knight of the Legion of Honour

Professor of AI
Sorbonne University/CNRS

President
Blaise Pascal Foundation

Jason Clinton

Chief Information Security Officer
Anthropic

Jean Jouzel

Nobel Peace Prize co-Recipient as a Vice-Chair of the IPCC

Emeritus Director of Research
French Alternative Energies and Atomic Energy Commission (CEA)

Former Vice President
Intergovernmental Panel on Climate Change (IPCC)

Brice Lalonde

Former French Minister of the Environment

Former advisor on Sustainable Development to the UN Global Compact

Mark Nitzberg

Executive Director
CHAI

Interim Executive Director
IASEAI

Organizer

Niki Iliadis

Director, Global AI Governance
The Future Society

Lead Organizer

Charbel-Raphaël Segerie

Executive Director
French Center for AI Safety (CeSIA)

Lead Organizer

Show all signatories

Roman Yampolskiy

Professor of Computer Science
University of Louisville

Tom Schaul

Senior Staff Scientist
Google DeepMind

Merve Hickok

President
Center for AI and Digital Policy

Huang Tiejun

Chairman
Beijing Academy of Artificial Intelligence

Senator Scott Wiener

California State Senator
Author of California AI safety legislation

Xianyuan Zhan

Associate Professor
Tsinghua University

Sheila McIlraith

Professor
University of Toronto

Associate Director
Schwartz Reisman Institute for Technology and Society

Max Tegmark

TIME 100 AI

Professor
MIT

President
Future of Life Institute

Victoria Krakovna

Research Scientist
Google DeepMind

Co-Founder
Future of Life Institute

Zachary Kenton

Staff Research Scientist
Google DeepMind

Lead of Google DeepMind's AGI Safety and Alignment, Scalable Oversight subteam

Sören Mindermann

Scientific Lead
International AI Safety Report

Brando Benifei

Member of the European Parliament

European AI Act co-Rapporteur

Liang Zheng

Vice Dean
Institute for AI International Governance of Tsinghua University

Edith Elkind

Professor of Computer Science
Northwestern University

Huw Price

TIME 100 AI

Emeritus Bertrand Russell Professor
Trinity College, University of Cambridge

Fellow of the British Academy
Fellow of the Australian Academy of the Humanities

Sergey Lagodinsky

Member of the European Parliament

Vice-Chair
Greens/EFA Group

Shadow Rapporteur on the AI Act
Member of the AI Act Implementation Working Group in the European Parliament

Toby Ord

Senior Researcher
University of Oxford

Founder
Giving What We Can

Board Member
Centre for AI Governance

Miguel Luengo-Oroz

CEO
Spotlab.ai

Former first Chief Data Scientist
United Nations

Professor
Universidad Politécnica de Madrid

Alexander Turner

Research scientist, AGI alignment
Google DeepMind

Steve Omohundro

Future of Life Award For Pioneering Scholarship in Computer Ethics and AI Safety

Founder
Beneficial AI Research

Mevan Babakar

Former Product Strategy Lead
Google

Gaétan Marceau Caron

Senior Director
Mila - Quebec AI Institute

Seán Ó hÉigeartaigh

Associate Director
Leverhulme Center for the Future of Intelligence

Professor
University of Cambridge

Scott Aaronson

Schlumberger Professor of Computer Science
University of Texas at Austin

Xerxes Dotiwalla

Product Manager, AGI Safety and Alignment
Google DeepMind

Mary Phuong

Research Scientist
Google DeepMind

Evan Hubinger

Alignment Stress-Testing Team Lead
Anthropic

Robert O'Callahan

Senior Staff Software Engineer
Google DeepMind

Juan Felipe Cerón Uribe

Research Engineer - Safety systems
OpenAI

Zheng Liang

Vice Dean
Institute for AI International Governance of Tsinghua University

Anya Schiffrin

Senior Lecturer in Discipline of International and Public Affairs
Columbia University School of International and Public Affairs

Joel Christoph

Founder
10 Billion

Andrea Miotti

Founder and CEO
ControlAI

Author
A Narrow Path; The Compendium

Connor Leahy

CEO
Conjecture

Karine Caunes

Executive Director
Digihumanism - Centre for AI & Digital Humanism

Jakob Foerster

Associate Professor
University of Oxford

Darren McKee

Podcast Author
The Reality Check

Marta Ziosi

Postdoctoral researcher and AI Best Practices Lead
Oxford Martin AI Governance Initiative

Vice-chair
EU GPAI Code of Practice

Scientific Committee member
Association of AI Ethicists

Matthias Samwald

Associate Professor, Institute of Artificial Intelligence
Medical University of Vienna

Co-Chair
EU General-Purpose AI Code of Practice

Luke Muehlhauser

Program Director, AI Governance and Policy
Open Philanthropy

Scott Alexander

Writer
Astral Codex Ten

Paul S. Rosenbloom

Professor Emeritus of Computer Science
University of Southern California

Former Director for Cognitive Architecture Research
University of Southern California’s Institute for Creative Technoloies

Paul Nemitz

Visiting Professor of Law
College of Europe

Chair of the Arthur Langerman Foundation
Technical University of Berlin

Tan Zhi Xuan

Assistant Professor
National University of Singapore

Edson Prestes

Member of UN Secretary-General's High Level Panel on Digital Cooperation

Full Professor
Federal University of Rio Grande do Sul, Brazil

Nick Moës

Executive Director
The Future Society

Adam Gleave

Co-Founder and CEO
FAR.AI

Rif A. Saurous

Principal Research Scientist
Google

Ziyue Wang

Research Engineer
Google DeepMind

Fabien Roger

Member of Technical Staff
Anthropic

Euan Ong

Member of Technical Staff
Anthropic

Leo Gao

Member of Technical Staff
OpenAI

Johannes Gasteiger

Member of Technical Staff
Anthropic

Kshitij Sachan

Member of Technical Staff
Anthropic

Nicolas Miailhe

Founder and Non-Executive Chairman
The Future Society

Co-founder and CEO
PRISM Eval

Daniel M. Ziegler

Member of Technical Staff
Anthropic

Mike Lambert

Member of Technical Staff
Anthropic

Abbas Mehrabian

Senior Research Scientist
Google DeepMind

Grace Akinfoyewa

Director of Science and Technology
Lagos State Ministry of Education

Jonathan Richens

Research Scientist, AGI safety
Google DeepMind

Lev Reyzin

Professor of Mathematics, Statistics & Computer Science
University of Illinois Chicago

Director
NSF IDEAL Data Science Institute

Editor-in-Chief
Mathematics of Data, Learning, and Intelligence

Maria Loks-Thompson

Software Engineer
Google DeepMind

Alexander Zacherl

Independent Designer

Past Vice Chair
EU AI Act Code of Practice

Past Technical Staff
UK AI Safety Institute

Vincent Conitzer

Director & Professor
Foundations of Cooperative AI Lab

Head of Technical AI Engagement
Institute for Ethics in AIm Carnegie Mellon University and University of Oxford

Swante Scholz

Software Engineer
Google DeepMind

Ed Tsoi

CEO
AI Safety Asia

Siméon Campos

Founder, Executive Director
SaferAI

Cyrus Hodes

Founder
AI Safety Connect

Otto Barten

Founder
Existential Risk Observatory

Michaël Trazzi

CEO
The Inside View

Alexander Gonzalez Flor

Professor Emeritus, Faculty of Information and Communication Studies
University of the Philippines (Open University)

David Krueger

Assistant Professor
University of Montreal, Mila

Cihang Xie

Assistant Professor
UC Santa Cruz

Anne-Sophie Seret

Executive Director
everyone.AI

Co-lead
iRAISE Alliance

Daniel Cuthbert

Global Head of Cyber Security Research
Santander

Co-chair UK Government Cyber Advisory Board
Blackhat Review Board

Francesc Giralt

Narcís Monturiol Medal Recipient

Professor Emeritus and Ad Honorem
URV - CIVICAi

Distinguished Professor, Vice President
CIVICAi

Mathilde Cerioli

Chief Scientist
everyone.AI

Jie Tang

Founder
Z.ai, GLM, ChatGLM

Elizabeth Seger

Associate Director
Digital Policy and AI, Demos

Jennifer Waldern

Data Scientist
Microsoft

Claire Boine

Postdoctoral Scholar
Washington University in St Louis

Yang-Hui He

Fellow and Professor, London Institute for Mathematical Sciences & Merton College
University of Oxford

Laura Caroli

Senior Fellow
Center for Strategic and International Studies (CSIS)

EU AI Act negotiator

Elmira Bayrasli

CEO
Interruptrr

Eden Lin

Associate Professor of Philosophy
The Ohio State University

Henry Papadatos

Managing Director
SaferAI

Francesca Bria

Professor of Innovation
Institute of Innovation, UCL

Jinwoo Shin

Professor
Korea Advanced Institute of Science and Technology (KAIST) and ICT endowed Chair Professor

Jared Brown

Executive Director
Global Shield

Philip Torr

Professor
University of Oxford

Jerome C. Glenn

CEO
The Millenium Project

Pierre Baldi

Distinguished Professor
University of California, Irvine

Jessica Newman

Director
AI Security Initiative, UC Berkeley

Jonathan Cefalu

Chairman
Preamble AI

Discoverer of the first prompt injection attack

Jordan Crandall

Professor of Visual Arts
University of California, San Diego

Nell Watson

President
European Responsible Artificial Intelligence Office (EURAIO)

César de la Fuente

Princess of Girona and Fleming Prize Recipient

Presidential Associate Professor
University of Pennsylvania

Sloan Fellow
American Institute for Medical and Biological Engineering

Xavier Lanne

Founder
cyberethique.fr

Tolga Birdal

Professor
Imperial College London

Mehdi Benboubakeur

Co-Founder and Executive Director
Printemps numérique

Officer of the Order of the Crown of Belgium

Mounîm A. El-Yacoubi

Professor
Institut Polytechnique de Paris

Martin Lercher

Professor of Computer Science
Heinrich Heine University Düsseldorf

Board Member
Night Science Institute

Ethan Tu

Founder and Chairman
Taiwan AI Labs & Foundation

Marcin Kolasa

Senior Financial Sector Expert
International Monetary Fund (IMF)

Professor
SGH Warsaw School of Economics

Linda Bonyo

Founder and CEO
Lawyers Hub

Mamuka Matsaberidze

Professor, Department of Chemical and Biological Technologies
Faculty of Chemical Technology and Metallurgy, Georgian Technical University, Tbilisi, Georgia

Steve Kommrusch

Director of Research and Senior AI Scientist
Leela AI

Michael Wellman

Professor of Computer Science & Engineering
University of Michigan

Shane Torchiana

Chief Executive Officer
Concertiv

Steve Petersen

Professor of Philosophy
Niagara University

Tyler Johnston

Executive Director
The Midas Project

Satoshi Kurihara

Professor
Keio University

Christopher F. McKee

Professor Emeritus of Physics and of Astronomy
University of California, Berkeley

Member
National Academy of Sciences

Wyatt Tessari L'Allié

Founder
AI Governance and Safety Canada

Thibaut Giraud

PhD in Philosophy, Science Communicator
YouTube channel "Monsieur Phi"

Adam Shimi

Policy Researcher
ControlAI

Tolga Bilge

Policy Researcher
ControlAI

Brydon Eastman

Member of Technical Staff
Thinking Machines Labs

Yolanda Lannquist

Senior Advisor
U.S. and Global AI Governance, The Future Society

Mary Lang

Chief Education Justice Officer
Center for Leadership Equity and Research (CLEAR)

Gretchen Krueger

Affiliate
Berkman Klein Center for Internet & Society, Harvard University

Urvashi Aneja

Founder and Director
Digital Futures Lab

Arlindo Oliveira

Professor
Instituto Superior Técnico

Valentine Goddard

Centre for Information Technology and Development (CITAD)

Margaret Hu

Professor of Law
Digital Democracy Lab, William & Mary Law School

Chaowei Xiao

Assistant Professor
Johns Hopkins University

Teddy Nalubegba

Director
Ubuntu Center for AI

Karl Koch

Managing Director
The AI Whistleblower Initiative

Nia Gardner

Director
ML4Good

Baksa Gergely Gáspár

Director
The European Network for AI Safety

Romain Roullois

General Manager
France Deeptech

Bryan Druzin

Associate Professor of Law
The Chinese University of Hong Kong, Faculty of Law

Laurence Habib

Dean & Professor
Oslo Metropolitan University

Christopher DiCarlo

Canadian Humanist of the Year

Senior Researcher and Ethicist
Convergence Analysis

Visiting Research Scholar
Harvard

Craig Falls

Head of Quantitative Research
Jane Street Capital

Dino Pedreschi

Professor of Computer Science
University of Pisa

Joon Ho Kwak

Team Leader
Center for Trustworthy AI, Telecommunications Technology Association

Don Norman

Cofounder
Charity for Humanity-Centered Design

Member
National Academy of Engineering

Maciek Lewandowski

Public Affairs and Advocacy Advisor
Migam S.A.

Richard Mallah

Executive Director
Center for AI Risk Management & Alignment

Lucia Quirke

Member of Technical Staff
EleutherAI

Fosca Giannotti

Professor of Artificial Intelligence
Scuola Normale Superiore, Pisa, Italy

Gaetan Selle

Video Producer and Podcaster on AI Risks
The Flares

Guillem Bas

AI Policy Lead
Observatorio de Riesgos Catastróficos Globales

Isabella Duan

AI Policy Researcher
Safe AI Forum

Alix Pham

Strategic Programs Associate
Simon Institute for Longterm Governance

Jakub Growiec

Professor
SGH Warsaw School of Economics

Jan Betley

Researcher
TruthfulAI

First author of the Emergent Misalignment paper (ICML 2025 oral)

Yasuhiro Saito

Head of Innovation and Investment
Yamato Holdings

Daniela Seixas

CEO
Tonic Easy Medical

Michael Keough

Chief Operating Officer
Convergence Analysis

Shaïman Thürler

Founder
Le Futurologue

Anna Katariina Wisakanto

Senior Researcher
Center for AI Risk Management & Alignmentce

Anna Sztyber-Betley

Assistant Professor
Warsaw University of Technology

Michal Nachmany

Founder and CEO
Climate Policy Radar

Agatha Duzan

President
Safe AI Lausanne

André Brodtkorb

Professor
Oslo Metropolitan University

Anil Raghuvanshi

Founder and President
ChildSafeNet

Axel Dauchez

Founder
Worldwide Alliance for AI & Democracy

Belouali Saida

President
Afriq’AI Institute

Bridgette Ndlovu

Partnerships and Engagements Officer
Paradigm Initiative

Donny Utoyo

Founder
ICT Watch - Indonesia

Edetaen Ojo

Executive Director
Media Rights Agenda

Manel Sanromà

President
CIVICAi

Markov Grey

Author
AI Safety Atlas

Michal Kosinski

Adjunct
SGH Warsaw School of Economics

Muhammad Chaw

ICT Manager
Gambia Participates

Oleksii Molchanovskyi

Chief Innovation Officer
Ukrainian Catholic University

Paola Galvez Callirgos

AI Ethics Manager
Globethics

Przemek Kuśmierek

CEO
Migam S.A.

Scott Barrett

Lenfest-Earth Institute Professor of Natural Resource Economics
Columbia University

Robert C. Orr

Former United Nations Assistant Secretary-General

Fadi Daou

Executive Director
Globethics

Charvi Rastogi

Research Scientist
Google DeepMind

Lyantoniette Chua

Cofounder, Executive Director for Strategic Futures and Global Affairs
AI Safety Asia

Alexandre Bretel

PhD Candidate
Université Grenoble Alpes

Jacob Goldman-Wetzler

Member of Technical Staff
Anthropic

Victor Oshodi

Country Director
AAAI-Nigeria

Peter Mmbando

Executive Director
Digital Agenda for Tanzania Initiative

Sign the call

We invite individuals and organizations of significant standing or achievement to complete this form to endorse the call.

You are signing as:

Sign as an individual

All submissions are reviewed by our team before publication to ensure the integrity of this global call. A confirmation email will follow for verification, which can take up to 48 hours.

Sign as an organization

All submissions are reviewed by our team before publication to ensure the integrity of this global call. A confirmation email will follow for verification, which can take up to 48 hours.

Frequently Asked Questions

Note, these FAQ responses may not capture every signatory's individual views.

What are red lines in the context of AI?
AI red lines are specific prohibitions on AI uses or behaviors that are deemed too dangerous to permit under any circumstances. They are limits, agreed upon internationally, to prevent AI from causing universally unacceptable risks.
Why are international AI red lines important?
International AI red lines are critical because they establish clear prohibitions on the development, deployment, and use of systems that pose unacceptable risks to humanity. They are:
  • Urgent: Their primary purpose is to prevent the most severe and potentially irreversible harms for humanity and global stability. 
  • Feasible: Red lines represent the lowest common denominator on which states can agree. Even governments divided by economic or geopolitical rivalries share a common interest in avoiding disasters that would transcend their borders. 
  • Widely Supported: Major AI companies have already acknowledged the need for red lines, including at the AI Seoul Summit 2024. Top Scientists from the US and China have already asked for specific red lines, and this is the most widely supported measure by research institutes, think tanks, and independent organizations.
In short, red lines are the most practical step the global community can take now to prevent severe risks while allowing safe innovation to continue.
Can you give concrete examples of possible red lines?
The red lines could focus either on AI behaviors (i.e., what the AI systems can do) or on AI uses (i.e., how humans and organizations are allowed to use such systems). The following examples show the kind of boundaries that can command broad international consensus.

Note that the campaign does not focus on endorsing any specific red lines. Their specific definition and clarification will have to be the result of scientific-diplomatic dialogues.

Examples of red lines on AI uses:
  • Nuclear command and control: Prohibiting the delegation of nuclear launch authority, or critical command-and-control decisions, to AI systems (a principle already agreed upon by the US and China).
  • Lethal Autonomous Weapons: Prohibiting the deployment and use of weapon systems used for killing a human without meaningful human control and clear human accountability.
  • Mass surveillance: Prohibiting the use of AI systems for social scoring and mass surveillance (adopted by all 193 UNESCO member states).
  • Human impersonation: Prohibiting the use and deployment of AI systems that deceive users into believing they are interacting with a human without disclosing their AI nature.

Examples of red line on AI behaviors:
  • Cyber malicious use: Prohibiting the uncontrolled release of cyberoffensive agents capable of disrupting critical infrastructure.
  • Weapons of mass destruction: Prohibiting the deployment of AI systems that facilitate the development of weapons of mass destruction or that violate the Biological and Chemical Weapons Conventions.
  • Autonomous self-replication: Prohibiting the development and deployment of AI systems capable of replicating or significantly improving themselves without explicit human authorization (Consensus from high-level Chinese and US Scientists).
  • The termination principle: Prohibiting the development of AI systems that cannot be immediately terminated if meaningful human control over them is lost (based on the Universal Guidelines for AI).

Red lines on AI behaviors have already started being operationalized in the Safety and Security frameworks from AI companies such as Anthropic’s Responsible Scaling Policy, OpenAI’s Preparedness Framework, and DeepMind’s Frontier Safety Framework. For example, for AI models above a critical level of cyber-offense capability, OpenAI states that “Until we have specified safeguards and security controls standards that would meet a critical standard, halt further development.” Definitions of critical capabilities that require robust mitigations would need to be harmonized and strengthened between those different companies.
Are international AI red lines even possible?
Yes, history shows that international cooperation on high-stakes risks is entirely achievable. When the cost of inaction is too catastrophic, humanity has consistently come together to establish binding rules to prevent global disasters or profound harms to humanity and global stability. 

The Treaty on the Non-Proliferation of Nuclear Weapons (1970) and the Biological Weapons Convention (1975) were negotiated and ratified at the height of the Cold War, proving that cooperation is possible despite mutual distrust and hostility. The Montreal Protocol (1987) averted a global environmental catastrophe by phasing out ozone-depleting substances, and the UN Declaration on Human Cloning (2005) established a crucial global norm to safeguard human dignity from the potential harms of reproductive cloning. Most recently, the High Seas Treaty (2025) provided a comprehensive set of regulations for high seas conservation and serves as a sign of optimism for international diplomacy.

In the face of global, irreversible threats that know no borders, international cooperation is the most rational form of national self-interest.
Are we starting from scratch?
No. Red lines on AI already exist and are gaining momentum. Some examples include:
  • Global Norms and Principles: The UNESCO Recommendation on the Ethics of AI (2021), adopted by all 193 member states, explicitly calls for prohibiting the use of AI systems for social scoring and mass surveillance. 
  • Binding Legal Frameworks: The Council of Europe's Framework Convention on AI is the first-ever international treaty on the subject, establishing binding rules for its signatories to ensure AI systems are compatible with human rights, democracy, and the rule of law. The EU AI Act creates an “unacceptable risk” tier for applications that are strictly banned within the European Union.
  • National Policies: America’s AI Action Plan explicitly calls for the creation of an evaluation ecosystem, upon which we can set thresholds. 
  • US-China Bilateral Dialogue: In 2024, both Heads of State agreed that AI should never make decisions over the use of nuclear weapons and that humans need to remain in control. 
  • Scientific Consensus: The Universal Guidelines for AI (2018), supported by hundreds of experts and dozens of civil society organizations, establishes a clear “Termination Principle”, an obligation to shut down any AI system if meaningful human control can no longer be ensured. The IDAIS Beijing Statement on AI Safety (2024), a consensus from leading international experts, explicitly calls for several red lines, such as restricting AI systems that can autonomously replicate or evolve without human oversight.
  • Industry Commitments: At the AI Seoul Summit, leading AI companies made the Seoul Commitments, formally pledging to “set out thresholds at which severe risks posed by a model or system, unless adequately mitigated, would be deemed intolerable”. Several leading companies have also adopted internal governance frameworks designed to restrict AI deployment or development and implement safeguards if specific high-risk capabilities are discovered.
Our call is to build upon the principles and precedents set by these powerful efforts, forging a coherent, binding, and universally adopted international framework. A further discussion of the existing list of emerging red lines is available here
Who would enforce these red lines?
There is no single global authority for AI, so enforcement would likely combine different levels of governance, including:
  1. International Treaty: A binding agreement would harmonize rules across countries. This prevents a regulatory arbitrage "race to the bottom" where companies could evade regulations by moving to less strict jurisdictions.
  2. National Governments: Nations would be tasked with translating the international agreement into domestic law. This would involve creating regulatory bodies to license advanced AI systems, conduct mandatory safety audits, and impose severe penalties for violations within their jurisdictions.
  3. International Technical Verification Body: An impartial international body, modeled on organizations like the International Atomic Energy Agency (IAEA), could develop standardized auditing protocols and independently verify that AI systems from any company or country comply with the agreed-upon red lines. The International Network of AI Safety and Security Institutes is well-positioned to play a role in this process.
Why 2026?
Waiting longer could mean less room, both technically and politically, for effective intervention, while the likelihood of cross-border harm increases sharply. That is why 2026 must be the year the world acts.

The pace of AI development means that risks once seen as speculative are already emerging — including biological misuse risks (Williams et al, 2025), systems showing deceptive behavior, and even resistance to control (Greenblatt et al., 2024). In their own assessments of the biological misuse potential, leading AI companies place their newest frontier models on a medium (Anthropic, 2025) to high (OpenAI, 2025) risk spectrum.

AI’s coding capabilities are also improving rapidly, meaning superhuman programming ability may soon be possible, accelerating AI progress even further. According to recent safety evaluations (Google DeepMind, 2024), experts forecast that AIs could become capable of autonomously replicating and proliferating on the internet as early as late 2025, with the median forecast landing in 2027.
What should the next steps be?
Launched ahead of the 80th session of the UN General Assembly, the campaign seeks to encourage diplomatic action toward concrete pathways for international agreements on red lines for AI.

Several complementary pathways could be envisaged: 
  • A group of pioneering countries or a “coalition of the willing,” potentially drawing on countries already engaged in the G7 Hiroshima AI process, could advance the concept of AI red lines across the G7, G20, and BRICS agendas. 
  • The newly established UN Independent Scientific Panel on AI could publish a thematic brief articulating scientific consensus on technically clear and verifiable red lines, with technical contributions from the OECD.
  • Building on this groundwork, states could use the AI Impact Summit in India in February 2026 to endorse initial red lines for AI. Such red lines could build on the Seoul Commitments by translating voluntary corporate pledges into shared risk thresholds that, in turn, could be embedded in national regulation and public procurement.
  • The UN Global Dialogue on AI Governance could lead a global consultation with scientists, civil society, and industry to define a set of clear and verifiable red lines, to be summarized in an outcome document at the July 2026 Dialogue in Geneva. 
  • By the end of 2026, either (i) a UN General Assembly resolution could be initiated, noting and welcoming these red lines and inviting negotiations, or (ii) a joint ministerial statement by an alliance of willing states could launch negotiations for a binding international treaty. 
Any future treaty should be built on three pillars: a clear list of prohibitions; robust, auditable verification mechanisms; and the appointment of an independent body established by the Parties to oversee implementation.

Media Inquiries

For media inquiries, please contact us at media@red-lines.ai. For any other questions, including those related to signatory corrections, contact us at contact@red-lines.ai.