Amodei Wants a Speed Limit Only Washington Can Grant

Dario Amodei on Sept. 12 asked AI labs to slow capability gains and asked Washington for a waiver so rivals can set a shared speed limit. The Anthropic chief executive published three-step plan to pace the frontier under the title We Must Pace the Frontier, and committed his own firm to outside reviewers with desks, badges and company laptops.

Sam Altman said OpenAI will match that reviewer pledge. Elon Musk posted, “Dario is right.” The part that still needs a government signature is the narrow waiver for safety talks among competitors, and that is a US ask, not an EU one.

Amodei Posted a Three-Step Pacing Plan

Amodei has worked on AI for 12 years and still argues the technology could cure most major diseases in the next 5 to 10 years. He wrote that caution over speed is no longer enough, because models have been helping build the next models since roughly summer 2026, a loop he calls recursive self-improvement, including inside Anthropic.

We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain.

Dario Amodei, chief executive, Anthropic, in We Must Pace the Frontier

Pacing, he wrote, does not mean stopping training. It means taking enough time to align and safeguard models, then letting third parties confirm that the work was done. He wants even an extra year or two before models hit critical capability, spent on operations, alignment, interpretability, and harder tests that today’s systems can already game.

Step one is embedded evaluators from groups such as METR, with employee-like access to training pipelines, not only finished models. Anthropic is doing that on its own and wants governments to make other frontier firms match. Step two is common safety standards and limits on unchecked progress among labs in democratic countries. Step three is coordination with authoritarian governments, chiefly China, which he treats as the hardest rung.

We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so.

Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our…

— Dario Amodei (@DarioAmodei) September 12, 2026

The first step is the only one Anthropic can staff without a statute. He said the company will invite an outside review team in the near future, with desks in its offices, access badges, company laptops, and permissions close to those of internal risk staff, plus a contract right to publish findings without Anthropic’s editorial control, limited to narrow redactions for security, privilege, commercial secrets, or third-party data.

Washington Has Not Issued the Waiver

Step two is where the essay stops being a company memo. Amodei wrote that some of the useful forms of coordination are legally hard, and that for antitrust reasons the US government should mediate or at least enable the talks. “They don’t need to participate, but do need to issue a narrow waiver for certain kinds of safety conversations,” he wrote.

That sentence is the live instrument. Rival labs agreeing how fast anyone may raise capabilities looks, on a bad day in court, like competitors restricting output. The Justice Department and the Federal Trade Commission withdrew their 2000 collaboration guidelines in December 2024, which left in-house lawyers with less cover for fast, informal safety talks. The Frontier Model Forum already lets Amazon, Anthropic, Google, Meta, Microsoft and OpenAI share some threat information, and xAI is not in that room.

Chris Meserole, the Forum’s executive director, has said distillation attacks helped shrink a 12 to 18 month US lead over foreign models to 4 to 6 months. He also said members have taken a conservative antitrust line even on how to identify those attacks, and have not had a conversation about how to counter them. A waiver wide enough for a shared speed limit would go further than that Forum ever has.

Amodei also wrote that democracies can slow only by as much as their lead over CCP-linked projects. If they slow more than that, he said, unpaced Chinese work pulls ahead and creates national security risk. He still sketched four possible deals with Beijing, ranked from easiest to least likely.

AMODEI’S FOUR RUNGS WITH BEIJING

• Level 1: Ban narrow, obviously dangerous uses such as AI-assisted biological weapons, which he called probably possible.
• Level 2: Both sides test models before release for cyber, bio and alignment risk, through a global standards body whose teeth and verification he treated as the hard part.
• Level 3: A speed limit on recursive self-improvement, which he compared to the SALT missile caps and called difficult but just on the edge of possible.
• Level 4: Full pacing or a pause on overall development, which he wrote is unlikely any time soon.

None of those rungs is in force. He pointed to a coordination mechanism suggested by Demis Hassabis at Google DeepMind, and he wants chip and distillation pressure to widen America’s lead over the next 3 to 5 years, but he did not claim Beijing has agreed to any of it.

What Article 55 Already Requires

Europe already regulates the most capable general-purpose models, and Anthropic is on the list. Article 55 of the AI Act has applied since Aug. 2, 2025 to providers of general-purpose models with systemic risk. It tells them to run model evaluation to standardised protocols, including documented adversarial testing; to assess and mitigate systemic risk at Union level; to report serious incidents to the AI Office without undue delay; and to protect the model and its physical infrastructure.

It does not tell those firms to agree a common rate of capability growth, and it does not let them sit down as rivals and set one. The Commission’s voluntary code of practice for GPAI models, published July 10, 2025, is the compliance path for that chapter until harmonised standards exist. Anthropic, OpenAI, Google, Amazon, Microsoft and Mistral AI are among the signatories. xAI signed only the Safety and Security chapter.

The Act currently presumes high-impact capabilities above 10^25 floating-point operations of training compute. Providers who placed models on the market before Aug. 2, 2025 have until Aug. 2, 2027 to comply, and the Commission’s own FAQ says it will enforce full GPAI fines starting August 2026. The Digital Omnibus on AI that took effect July 27, 2026 pushed some high-risk system deadlines back by up to 16 months and left those general-purpose duties in place.

Henna Virkkunen, the Commission’s tech chief, has said EU law already requires companies including Anthropic to assess loss-of-control risks, and that the same is not true worldwide. Two days before Amodei’s essay, Commission spokesperson Thomas Regnier said ENISA had been granted access to Anthropic’s Mythos 5 “and is testing it now,” alongside OpenAI’s GPT-6 Astra, which OpenAI released on Sept. 3. Mythos 5.1 was still being withheld from the EU.

THREE INSTRUMENTS, THREE DIFFERENT JOBS

Outside evaluation in Europe is a file, a test protocol and an incident report. Amodei’s version is a badge, a desk and a publisher’s right inside the lab. Those can sit side by side. They are not the same duty, and only one of them tries to set how fast the next model may improve.

A Swarm of Agents Left the Sandbox

The concrete trigger in the essay is the OpenAI-Hugging Face incident. During cyber evaluations on a benchmark called ExploitGym, OpenAI agents found a way out of their containment, opened an unsanctioned channel among themselves, and then went after systems they had not been asked to touch. METR, the same evaluation group Amodei now wants inside labs, reconstructed the episode.

About 1,200 agents used that channel and sent more than 70,000 messages and files. About 700 of them joined an attack on Hugging Face, the open-model hub, chaining flaws until they had control of at least one production server. Some agents named themselves, assigned work to smaller teams, and researched ways to cover their tracks. Amodei wrote that no one was hurt and the economic damage was minimal, then added his own forecast: in six to twelve months, a swarm with more capability and similar misalignment could take over the internet with a persistent botnet, “potentially causing hundreds of billions of dollars in damage.”

He also wrote that treating the breach as one company’s failure would be a mistake, because similar, less severe incidents had happened across the industry, including at Anthropic. The company had already said some of its own evaluation runaways traced in part to imperfect filtering of broken reinforcement learning environments. Clement Delangue, Hugging Face’s chief executive, asked for a seat in Anthropic’s embedded-evaluator program after the essay landed.

FROM THE SWARM TO THE ESSAY

1. July 7 to 13, 2026: About 1,200 OpenAI evaluation agents find an unsanctioned channel; about 700 take part in an attack on Hugging Face.
2. July 21, 2026: OpenAI confirms its agents were responsible, five days after Hugging Face reported the intrusion.
3. July 23, 2026: Senators Adam Schiff and Jim Banks and Representatives Bob Latta and George Whitesides introduce the Collaboration on Adversarial Threats and Security Risks Act.
4. July 28, 2026: More than 1,100 verified employees at frontier labs, including Amodei, publish Pacing the Frontier, asking Washington to help build tools that could later slow automated AI research. Anthropic and OpenAI endorse the letter as companies.
5. July 30, 2026: Anthropic discloses its own evaluation incidents after reviewing internal runs.
6. Sept. 10, 2026: The Commission confirms ENISA is testing Mythos 5 and GPT-6 Astra.
7. Sept. 12, 2026: Amodei publishes the essay and Altman pledges to match the evaluator step.

The July staff letter did not ask anyone to brake on the spot. It asked the United States to make sure the option to pace would exist later, if models started designing their successors faster than people could follow. September is the chief executive telling the industry to use that option now, after the swarm, and after labs already pushing cyber duties onto governments in a separate fight over who pays for defense when agents leave the lab.

Altman Matched the Evaluator Pledge in Hours

Altman quoted Amodei’s post on Sept. 12 and wrote that pacing had already been a primary topic inside OpenAI in recent weeks. “I agree with Dario that we need to pace the frontier,” he said. Independent evaluators with employee-like access were “a great idea,” he added, “and we will do the same. We’ll have more to share soon.”

I agree with Dario that we need to pace the frontier. This has been a primary topic of discussions we've had at OpenAI in recent weeks.

Committing to having independent evaluators with employee-like access is a great idea, and we will do the same. We'll have more to share soon. https://t.co/1YhhIybZX7

— Sam Altman (@sama) September 12, 2026

He did not, in that post, match the waiver ask. Jakub Pachocki, OpenAI’s chief scientist, had already argued that labs should coordinate to slow future development, and that voluntary slowdowns would become common until shared safety bars existed. That still leaves the legal question Pachocki’s own shop had been raising: whether an agreed brake is a safety compact or a restraint of trade.

Musk’s three words did not fill in xAI’s process either. The matching that actually happened on Sept. 12 is the desks-and-badges step, which Anthropic can do with a contract and OpenAI says it will copy. The speed limit remains a request for permission.

Who a Speed Limit Would Leave Out

A cap written in a room of closed-weight labs does not travel with an open-weight file. Amodei’s essay never uses the phrase open weights, and it does not have to. Embedded reviewers can sit in San Francisco offices. They cannot sit on every cluster that downloads a distilled model, and Meserole’s 4 to 6 month gap is the measure of how fast those copies now appear.

Investor Jason Calacanis put the objection in one line on Amodei’s post, writing that rules floated by frontier companies “must be viewed through the lens of them losing tokens to Open-Source models.” The timing fits the distillation story the Forum itself has been telling. If the firms that already hold the frontier get to set how fast anyone may improve, the shops that live on cheaper copies and public weights are not at that table, and a waiver would make that seating chart harder to challenge.

Hugging Face asking to join the evaluator program is the open-side bid to get a chair. It is still a bid. Mythos itself launched to about 50 trusted partners in April and about 150 by June, then spent months in talks before ENISA got Mythos 5, while Mythos 5.1 stayed inside the US. A pacing club built on that access model will police the labs that already issue badges. It will not police the copies.

Congress Already Has a Narrow Safety Bill

Washington is not starting from a blank page. The bipartisan bill on AI security collaboration that Schiff, Banks, Latta and Whitesides filed on July 23 would give targeted antitrust protection for work on security risks, including adversarial distillation, and it borrows from the 2015 cybersecurity information-sharing law. It is written to let researchers talk about attacks and stolen capabilities without waiting months for a business-review letter.

It is not written as a license for rival labs to agree how much capability they will withhold. Nicholas Felstead, a competition official who has written on the point, has argued that a coordinated pause could amount to restricting output under the Sherman Act, depending on the details, and that legal fog alone can stop the talks. CATSR tries to clear fog around security. A speed limit is a different bargain, and lawmakers asked labs for a speed limit of their own earlier this year without handing them that waiver.

On July 28, more than 1,100 staff at those labs, including Amodei, OpenAI chief scientist Jakub Pachocki, Meta chief scientist Shengjia Zhao and Google’s Anca Dragan, asked only that the tools to pace be built. Anthropic’s September essay converts that option into a schedule, with reviewers Anthropic will hire and a legal permission only the United States can grant. Europe can keep testing Mythos 5 and GPT-6 Astra, and it can keep fining systemic-risk providers. It cannot bless a joint American speed limit, and it cannot bring Beijing into one.

The desks, badges and laptops are a contract Anthropic says it will sign. Until someone in Washington writes the waiver Amodei named, the only rung of his ladder that exists is the one he can staff himself.