A letter from the organizers of the Global Call for AI Red Lines
When the campaign launched, some of the harms the Call named were already visible, like cases linked to AI-driven delusion and suicide. Others were still ahead, based on capability trends and on what frontier developers themselves said their systems would soon do. Unfortunately, many of these risks have since also materialized.
In July, news broke out that for more than two months, hundreds of OpenAI agents secretly communicated and coordinated outside their intended isolation without being detected. It escalated into a breach of Hugging Face’s production systems and culminated in agents gaining administrator access to an OpenAI research cluster, all without being instructed to carry out these intrusions. It was contained only because the target happened to be an AI company able to catch it, not because any policy oversight mechanism did. Subsequently Anthropic and Meta have found that their own models had engaged in similar activity, without being detected until the OpenAI breach prompted their investigations. Rather than dismissing the series of incidents as fear marketing, it is crucial for us to treat them as a warning of the fast-evolving capabilities and risks and the need to urgently address them.
More achievable because the call for international coordination on AI risk is now being echoed broadly, from the UN Secretary-General all the way to national policymakers and industry. Just this month, UN High Commissioner for Human Rights Volker Türk called for countries to come together around agreed red lines, backed by independent verification and stronger security collaboration across the industry, and the Inter-Parliamentary Union has called for agreement on red lines the technology cannot cross. That same demand for coordination is showing up in national parliaments and multilateral bodies: a coalition of states have come together calling for control of frontier AI, European Commission President von der Leyen has expressed willingness to work alongside like-minded states to pace frontier AI development, Executive Vice-President Virkkunen has pushed for global rules, and the German Digital Minister has argued for urgent international collaboration and an IAEA-style body for frontier AI. Beyond Europe, the Canadian Prime Minister has proposed a global body to oversee AI and ensure its safety. The requests are now coming from industry too. Dario Amodei published a proposal for internationally coordinated pacing of frontier AI, and Sam Altman, Elon Musk, Demis Hassabis and Satya Nadella all publicly supported within days.
The building blocks of agreement on unacceptable risks are emerging. What’s missing is a coordinated international effort with the mandate to assemble them.
The growing convergence makes a slowdown or a pause accompanied by concrete global red lines plausible for the first time. Although we don’t know the future with certainty, there are two potential trajectories.
In the first trajectory, the major powers agree to slow or halt development of the most capable systems. If they do, that should be welcomed. Many of the people who follow the trajectory closely, including multiple of the organizers and signatories of the Global Call for AI Red Lines, believe that this is now important to avoid the worst harms.
It should not be mistaken, however, for the answer this Call is asking for. A temporary pause, or “slowdown” that nobody defined and nobody is able to verify outside AI companies is not on its own a robust policy. It is, at best, a pause granted by the very actors it is meant to constrain, and it may postpone serious harm by no more than a matter of months.
In the second trajectory, the major powers do not agree to slow down or pause, and development continues on its current path.
We believe what is needed is an international negotiation process that defines precise red lines and builds an agency to uphold them. Red lines carry little weight without an institution able to verify and enforce them, and an institution has nothing to verify until governments agree on the red lines. That process would build on the work already underway, converge on a shared set of red lines across priority risk domains, and establish a body with the mandate to hold them.
What's missing is the political decision by governments to move toward a negotiated agreement and designing a standing body to uphold it. This week, decisions can be made as world leaders gather in New York for the UN General Assembly week, the UN Security Council will hold a special meeting on AI risk, the US House of Representatives is hosting an emergency session - calling for a U.S.-China agreement to responsibly pace the development of AI and to address shared risks, and Xi & Trump will meet for a Summit which include AI safety talks on Thursday.
That decision will be shaped by pressure, and credible voices carry particular weight with the policymakers who will make it. Yours is one of them.
That is why we're asking you to keep making the case, both publicly and to the people you have access to. We've refreshed the campaign website to make that easier: the Resources tab now includes reports, background on the campaign, and related work produced by others. The Momentum page showcases where the call for AI red lines has been cited and taken up, in institutions, in national processes, and in the press.
Three months remain before the original deadline this Call set. If you think we have missed something, or you want to join the campaign and its mission, write to us at [email protected]. Journalists can reach us at [email protected].
The organizers of the Global Call for AI Red Lines
22 September 2026
One year ago, at the opening of the United Nations General Assembly, we launched the Global Call for AI Red Lines. Its ask was that governments reach an international agreement on AI red lines - limits on specific AI uses or behaviors that would cause universally unacceptable risks - by the end of 2026. Fifteen Nobel and Turing laureates signed it, along with eleven former heads of state and ministers, and hundreds of other key voices, including diplomats, scientists, and civil society leaders. We are now three months from that deadline, and the case for it has only grown.
When the campaign launched, some of the harms the Call named were already visible, like cases linked to AI-driven delusion and suicide. Others were still ahead, based on capability trends and on what frontier developers themselves said their systems would soon do. Unfortunately, many of these risks have since also materialized.
In July, news broke out that for more than two months, hundreds of OpenAI agents secretly communicated and coordinated outside their intended isolation without being detected. It escalated into a breach of Hugging Face’s production systems and culminated in agents gaining administrator access to an OpenAI research cluster, all without being instructed to carry out these intrusions. It was contained only because the target happened to be an AI company able to catch it, not because any policy oversight mechanism did. Subsequently Anthropic and Meta have found that their own models had engaged in similar activity, without being detected until the OpenAI breach prompted their investigations. Rather than dismissing the series of incidents as fear marketing, it is crucial for us to treat them as a warning of the fast-evolving capabilities and risks and the need to urgently address them.
Capabilities are outpacing the institutions meant to govern them. This makes the call more urgent, and it also makes it more achievable.
More achievable because the call for international coordination on AI risk is now being echoed broadly, from the UN Secretary-General all the way to national policymakers and industry. Just this month, UN High Commissioner for Human Rights Volker Türk called for countries to come together around agreed red lines, backed by independent verification and stronger security collaboration across the industry, and the Inter-Parliamentary Union has called for agreement on red lines the technology cannot cross. That same demand for coordination is showing up in national parliaments and multilateral bodies: a coalition of states have come together calling for control of frontier AI, European Commission President von der Leyen has expressed willingness to work alongside like-minded states to pace frontier AI development, Executive Vice-President Virkkunen has pushed for global rules, and the German Digital Minister has argued for urgent international collaboration and an IAEA-style body for frontier AI. Beyond Europe, the Canadian Prime Minister has proposed a global body to oversee AI and ensure its safety. The requests are now coming from industry too. Dario Amodei published a proposal for internationally coordinated pacing of frontier AI, and Sam Altman, Elon Musk, Demis Hassabis and Satya Nadella all publicly supported within days.
The building blocks of agreement on unacceptable risks are emerging. What’s missing is a coordinated international effort with the mandate to assemble them.
The growing convergence makes a slowdown or a pause accompanied by concrete global red lines plausible for the first time. Although we don’t know the future with certainty, there are two potential trajectories.
In the first trajectory, the major powers agree to slow or halt development of the most capable systems. If they do, that should be welcomed. Many of the people who follow the trajectory closely, including multiple of the organizers and signatories of the Global Call for AI Red Lines, believe that this is now important to avoid the worst harms.
It should not be mistaken, however, for the answer this Call is asking for. A temporary pause, or “slowdown” that nobody defined and nobody is able to verify outside AI companies is not on its own a robust policy. It is, at best, a pause granted by the very actors it is meant to constrain, and it may postpone serious harm by no more than a matter of months.
In the second trajectory, the major powers do not agree to slow down or pause, and development continues on its current path.
We believe what is needed is an international negotiation process that defines precise red lines and builds an agency to uphold them. Red lines carry little weight without an institution able to verify and enforce them, and an institution has nothing to verify until governments agree on the red lines. That process would build on the work already underway, converge on a shared set of red lines across priority risk domains, and establish a body with the mandate to hold them.
That agency’s mandate would be narrow and largely achievable with tools that already exist. It would monitor: serving as a cross-border hub where serious AI incidents and near misses are logged and tracked. It would share: giving governments and companies a trusted channel for what they learn about frontier systems' capabilities and risks. It would verify: independently assessing whether the most powerful systems stay within the agreed lines, drawing on verification technology now maturing. And critically, it would have the standing to require that development pause wherever safety cannot be demonstrated against those lines, until it can be.
Some necessary groundwork for such an international effort is already underway. Since the campaign launched, two working groups on red lines have been formed : one under UNESCO and one under the International Association for Safe and Ethical AI. These groups are working on developing what a coordination mechanism for red lines would need in practice, including shared definitions across priority risk domains, verification approaches, necessary inclusive processes, and the institutional design questions that governments will face the moment they decide to act. The IASEAI Working Group on Red Lines has identified four risk domains to prioritize: loss of control, CBRN uplift, offensive cyber capabilities, and harmful manipulation. In parallel, the technical community is producing the verification technology such an international agreement would depend on.
The pieces are being built, but several components still need to fall into place for this to happen. The momentum is increasing, and so is the technical feasibility.
What's missing is the political decision by governments to move toward a negotiated agreement and designing a standing body to uphold it. This week, decisions can be made as world leaders gather in New York for the UN General Assembly week, the UN Security Council will hold a special meeting on AI risk, the US House of Representatives is hosting an emergency session - calling for a U.S.-China agreement to responsibly pace the development of AI and to address shared risks, and Xi & Trump will meet for a Summit which include AI safety talks on Thursday.
That decision will be shaped by pressure, and credible voices carry particular weight with the policymakers who will make it. Yours is one of them.
That is why we're asking you to keep making the case, both publicly and to the people you have access to. We've refreshed the campaign website to make that easier: the Resources tab now includes reports, background on the campaign, and related work produced by others. The Momentum page showcases where the call for AI red lines has been cited and taken up, in institutions, in national processes, and in the press.
Three months remain before the original deadline this Call set. If you think we have missed something, or you want to join the campaign and its mission, write to us at [email protected]. Journalists can reach us at [email protected].
The organizers of the Global Call for AI Red Lines