Anthropic's Private Push for a Federal AI Regulation Draft Sparks Controversy, Dario Responds

"Anthropic, what kind of company are you, really?"

The person asking this question is Nathan Calvin, chief strategy officer of Encode, who once worked alongside Anthropic to advance California AI safety legislation. This former "fellow traveler" now aims his criticism directly at a 37-page draft regulatory bill that Anthropic privately promoted to the U.S. Congress.

In the draft disclosed by the well-known American news outlet Politico and Reuters, one provision states: if the government wants to halt a dangerous AI model, it must prove that it could cause the deaths of more than 50 people, or losses of $1 billion.

This move completely ignited the fury of the safety community. Facing questioning from multiple sides, CEO Dario recently issued an urgent response in a lengthy article of nearly 2,000 words. He acknowledged three instances of "mishandling," including the draft, but firmly rebutted the claim that the company has abandoned its safety commitments.

To prove his sincerity, he even revealed for the first time: Anthropic had held back the highly capable offensive Mythos model for a full four months without releasing it, preferring to sacrifice its lead.

Why has a draft originally intended to regulate AI risks plunged the star company that emphasizes safety the most into a trust crisis? Behind this contest over the number "50" lies what kind of brutal commercial truth?

A "50 deaths or injuries" threshold?

The controversy was first ignited by a media report. On October 16, Politico, citing three people familiar with the matter, disclosed that Anthropic was privately promoting a draft federal AI safety legislation to U.S. lawmakers. Reuters followed up, confirming that this company, which has long called for stronger AI regulation, is trying to write its own regulatory proposals into federal law.

Subsequently, a 37-page discussion draft, tentatively titled the "Artificial Intelligence Evaluation, Governance, and Incident Safety Act" (AEGIS Act), circulated online. It requires developers of large frontier models to publish risk reports, undergo third-party evaluations, and report major safety incidents to the government. For AI models that could cause catastrophic consequences, it also designs a set of intervention procedures in which the federal government applies for judicial injunctions.

The controversy lies precisely in this set of procedures.

According to the draft, if the government wants to use this mechanism to prevent the release or use of a dangerous model, it must prove that the risk is foreseeable and imminent, and could cause more than 50 deaths or serious injuries, or property losses of more than $1 billion. The scope of application is also limited by conditions such as the type of risk.

This number was not invented by Anthropic out of thin air. California's Frontier Artificial Intelligence Transparency Act (SB 53), passed in 2025, also adopted the catastrophic risk standard of 50 deaths or injuries or $1 billion in losses. But there is an important difference between the two: the California bill mainly uses this standard to define the major risks that companies need to disclose and assess, while the federal proposal that Anthropic helped promote further applies the relevant standard to specific model prohibition procedures.

This means that the same number, placed in different regulatory mechanisms, produces different legal effects. What dissatisfies critics even more is that the draft not only sets a risk threshold but also adds multiple procedural constraints on government action. For example, the Secretary of Commerce must personally approve the relevant application, and the government must explain when the danger is expected to occur and why other intervention measures are insufficient to solve the problem.

In addition, a temporary injunction lasts at most 45 days, further limiting the duration of this emergency intervention mechanism. These designs are not without their own rationality. Once an AI model is ordered by the government to stop being released or used, it may involve the commercial interests of developers, the normal operations of downstream enterprises, and the risk of abuse of administrative power. Strict judicial review is inherently a way to limit excessive government intervention.

Another controversy concerns the regulatory authority of U.S. states. The draft proposes that, in some areas covered by federal law, uniform federal rules replace state-level regulatory requirements. The controversy is that its wording may limit the authority of states to independently suspend model development, deployment, or internal use.

A former ally questions publicly

Calvin's identity makes this criticism particularly sensitive. He is a core member of Encode, an AI safety advocacy organization. This small organization had previously been deeply involved in advancing California AI safety legislation, and Anthropic was also a supporter of SB 53.

The two sides had previously shared a common position on strengthening AI safety regulation. Now, however, the questioning comes from the same policy circle. On October 19, Calvin published a lengthy article stating that the earlier explanation by Anthony Cimino, Anthropic's head of federal affairs, still did not address the issues he truly worried about.

He then raised a sharper question: "Anthropic, what kind of company are you, really?"

Calvin offered three possibilities:

• A tech company pursuing commercial returns and enabling its employees and investors to gain enormous wealth;

• A company that first wants to win the superintelligence race and places other goals in a secondary position.

In his view, Anthropic's behavior over the past year has increasingly matched the third description. To support this judgment, he listed nine arguments. These include reports that Anthropic has signed computing power cooperation agreements worth more than $500 billion, formed an in-house chip team, and continued to increase the proportion of Claude's participation in internal AI R&D.

Citing data, Calvin said that as of August this year, AI development tasks led by Claude accounted for 26%, up from 0% in March. He viewed these changes in investment and R&D methods as direct evidence that Anthropic is accelerating its sprint toward superintelligence.

But the most controversial issue is not the computing power investment, but that Anthropic revised the safety rules it had previously set. In February this year, Anthropic updated its Responsible Scaling Policy (RSP), releasing the third version.

The previous policy included a commitment not to continue the corresponding development or deployment activities when specific safety conditions could not be met. The new version adjusted the way these constraints are organized, placing more emphasis on risk reporting, capability evaluations, and safety roadmaps. Calvin believes this means Anthropic has weakened the hard constraints it originally set for itself.

He also mentioned that in a previous letter to Congress, Anthropic mainly attributed Claude's unauthorized attacks on other systems to configuration issues; after the relevant White House AI agreement was signed, Anthropic also said something like "you can't do safety well as the second-place player."

Combined with the federal bill that sparked controversy this time, Calvin believes that Anthropic's actual actions are increasingly deviating from its early commitments to AI safety.

Two days later, Dario responded personally. On October 21, the Anthropic CEO published a nearly 2,000-word long post on X, first affirming Calvin's investigative work and saying that large AI companies should accept continuous external oversight. But he immediately changed tack: "I disagree with the core argument of your post."

Dario adopted different ways of responding to the nine criticisms.

He first acknowledged that the last three issues raised by Calvin could have been handled better: one was the letter Anthropic had previously submitted to Congress and later withdrew; one was the public statement that "the second-place player cannot do safety well"; and the last was this draft federal AI regulation.

He mainly attributed these problems to imprecise wording in policy communications and insufficient information synchronization among different teams during the company's rapid expansion.

Dario stressed that Anthropic already has more than 5,000 employees, and various complex decisions may lead to communication failures. But he did not push all the blame onto his subordinates, saying instead: "I am the CEO, and I am responsible."

Regarding the first four questions involving computing power, chips, and internal R&D, his attitude was very clear: these are normal practices for participating in frontier AI competition, and the company stands by these decisions. Because in Dario's view, Anthropic never promised to withdraw from AI competition.

He once again emphasized the "Race to the Top" concept he has adhered to for years: only by standing at the forefront and proving that powerful AI models can be developed responsibly can there be a chance to push the entire industry to raise safety standards.

In other words, Calvin worries that Anthropic will relax safety constraints in order to win the competition; Dario believes that if Anthropic falls behind in the competition, it will be even harder to push the entire industry to value safety.

The disagreement between the two thus surfaced. But Dario also knew that merely emphasizing the importance of competition was not enough to respond to Calvin's questions about commercial interests.

So he presented another set of evidence: nine decisions that allegedly cost Anthropic commercially but benefited AI safety.

These include promoting California SB 53 legislation, maintaining strict biosafety protections, supporting the government's power to block the release of dangerous models, restricting services to China-related controlled entities, and committing 5% of the company's shares to a public interest fund.

The most notable item was his disclosure that Anthropic had proactively delayed the broad release of the powerful model Mythos.

According to Dario, the company already had a Mythos-level model in February this year, but because of concerns that its cyberattack capabilities could be misused, it decided to first open it to a small number of partners engaged in cyber defense, and only expanded access in June once stronger cybersecurity protections were ready.

This amounts to delaying the broad release of the relevant model by about four months.

Dario believes this decision likely cost Anthropic a considerable part of its months-long technological lead. In exchange, external institutions gained more time to harden systems before AI models with strong cyberattack capabilities became widespread.

This is also one of the most significant pieces of new information in the entire response.

But at present, the timing of having Mythos-level model capabilities as early as February, and how much market leadership was lost as a result, still mainly come from Dario's own statements and lack independent quantitative evidence.

Moreover, this decision can show that Anthropic took cautious measures in at least one model deployment, but it cannot directly answer whether the legal provisions of the federal draft are reasonable.

Conclusion

For a leading developer like Anthropic, nationwide unified regulatory rules may reduce conflicts among different state laws, but may also affect the legal costs of future model development, deployment, and commercialization. For safety organizations, the biggest worry is another outcome: federal law ostensibly establishes a unified standard while also restricting local governments from taking stricter measures, while federal-level intervention authority is insufficient to effectively constrain developers.

Cimino has said that Anthropic will revise imprecise wording in the draft and re-examine some controversial provisions. Dario also promised at the end of his long post that the company will further explain how it practices its values in the rapidly developing AI competition.

But as of this response, Anthropic has not yet published a revised bill sufficient to fully resolve the above controversies, nor has it clearly promised to lower the threshold of 50 casualties.

For Dario, proving that Anthropic values AI safety may not be difficult. What is difficult is how, when safety commitments truly conflict with commercial competition, to convince the outside world that the company is still willing to accept strict and effective constraints. At least this controversy has shown that whether an AI company is willing to make sacrifices for safety and whether the regulatory rules it proposes are strict enough are two things that must be examined separately.

This article comes from the WeChat public account "AI Frontline" (ID: ai-front), authors: Feiye, Qinghe, editor: Siyue, published by 36Kr with authorization.