←News detail

36Kr · AIUpdated Nov 3 08:52

Anthropic Updates Prospectus: Admits It Previously Failed to Timely Identify AI R&D Acceleration, and Bans Claude from Fully Autonomously Designing Next-Generation Models

AI summary

Anthropic submitted an updated prospectus (S-1/A) to the U.S. SEC, changing the first version's "may be unable to identify before R&D significantly accelerates" to "has previously failed to identify," acknowledging that AI R&D acceleration has already genuinely occurred internally. The filing discloses that by September of this year, more than 90% of code in the internal codebase was generated by Claude, the average daily code commits per engineer rose from 35 times the 2024 average to more than 45 times, and internal spending on calling Claude, at public list prices, grew more than 250-fold from January to September. Page 188 explicitly states that Claude will not be allowed to fully autonomously design and build next-generation frontier models without human oversight, and that it is considering requiring key decisions to be reviewed and signed off by humans.

Why read this

AI score 57

In its updated prospectus, Anthropic admits that it previously failed to timely identify R&D acceleration, and draws a red line prohibiting fully autonomous model building, allowing readers to understand the tension between its safety and compute expansion.

Topics

Tags

Body · AI translation

Just moments ago, Anthropic, which is racing toward a Nasdaq IPO, submitted an updated prospectus (S-1/A) to the U.S. SEC.

Only a week after the first version of the prospectus was made public, Anthropic's management stayed up overnight to add a "confession" to the filing that makes one's heart tighten—

The red line for AI self-acceleration is close at hand. Internal estimates show that future Claude models will soon trigger the "AI significantly accelerates R&D" threshold, completing in just 1 year the R&D progress that previously took a full 2 years!

Even the people building AI are looking in the rearview mirror. Hampered by "lagging indicators," Anthropic admitted for the first time that it has in the past failed to detect R&D acceleration in time, and that in the future it may also only realize acceleration has occurred after the fact!

It explicitly prohibits "fully automated AI building AI." Even though more than 90% of internal code has already been taken over by Claude, on page 188 the company still draws a bottom line: it will never detach from human oversight and let Claude fully automatically design the next generation of frontier models!

Six words rewrite the prospectus

Putting the two versions of the prospectus side by side and comparing them line by line, the most critical change is on page 29.

In the first version of the S-1 a week ago, Anthropic still left room in its wording, saying only that the company "may be unable" to identify significant acceleration in R&D before it occurs.

In this revised version, legal and management specifically inserted six English words into that sentence: "have in the past not identified."

In just six words, a warning that "it may happen in the future" was turned directly into a confession that "it has already happened in the past."

In Anthropic's own labs, the situation in which AI R&D suddenly speeds up while the human team only reacts after the fact has already really occurred!

According to internal estimates, future models will soon hit the "self-acceleration red line" in the Responsible Scaling Policy.

Under the definition of this red line, once it is crossed, it means that with AI's help, a full 2 years of R&D progress at the 2018-2024 baseline pace can be completed in just 1 year.

The reason even top researchers who personally write algorithms and monitor data end up "realizing it only afterward" is that it is very difficult to measure improvements in large model capabilities accurately in real time.

Exactly how much stronger a new model has become in pretraining and agent collaboration cannot be read off the training curves alone.

Researchers must wait until the model finishes training, completes evaluations, and is then brought into the internal engineering pipeline for several months of practical use, and finally tally code output and experiment cycles, before they can back out the true acceleration multiple.

But that is where the time lag lies.

By the time researchers use months of lagging data to calculate that "it has already crossed the line," the new model has long since been writing the core code for the next-generation model day and night without stopping.

Internal token bills surpass total payroll

yet it dares not let Claude fully automatically build the next generation

The reason observation cannot keep up is that Claude has taken over Anthropic's internal R&D far too quickly.

The updated prospectus disclosed a set of the latest internal third-quarter data. By September of this year, more than 90% of the code in Anthropic's internal codebase had come from Claude.

Driven by AI, the amount of code each engineer submits per day soared in just one quarter from 35 times to more than 45 times the 2024 average.

Internal compute consumption has risen even more sharply. From January to September of this year, the company's internal spending on calling Claude, calculated at public list prices, surged by more than 250 times.

As early as April of this year, Anthropic's monthly spending on internal Claude had already exceeded the total monthly cash salaries paid to all employees (excluding equity incentives).

Logically, since Claude writes code dozens of times faster than humans and its bills have even surpassed human salaries, the most natural move would be to let go completely and have Claude fully automatically design and train the next generation of Claude.

Yet at precisely this moment, Anthropic itself drew a red line!

At the end of the chapter on the internal R&D loop on page 188, Anthropic specifically added a hard rule: even if it becomes technically entirely feasible in the future, the company will not detach from human oversight and understanding to let Claude fully automatically design and build the next generation of frontier models.

The concern behind this is that as models take over more and more R&D work, if some generation of model harbors vulnerabilities, biases, or deceptive tendencies that humans have not discovered, the risk will be passed down through the R&D process.

Once it is given free rein to fully automatically design the next generation, these hidden dangers will accumulate across generations and be amplified layer by layer, and by the time humans notice the problem, it will long since have become very difficult to trace the source.

To prevent this kind of cross-generational loss of control, Anthropic disclosed on page 31 that it is considering revising its safety red lines to mandate that specific key decisions in the R&D process must be reviewed and signed off by humans.

But holding the safety boundary has also made Anthropic pay a considerable price in R&D progress.

A few weeks ago, Dario himself revealed that "an already-trained advanced Mythos-class model was kept in cold storage for about four months."

Recently, in order to prevent model weights from being stolen and to strengthen security measures, the company delayed the start of pretraining for a brand-new frontier base model by more than a month.

For example, Mythos, the model with extremely strong vulnerability-discovery capabilities from April this year, has so far only been provided in a targeted way to a small number of institutions for defense through the "Project Glasswing" program.

Signing 895.4 billion in long-term compute contracts

whoever slows down is out

On one hand, it delayed pretraining of a new model by more than a month because of safety concerns; on the other hand, Anthropic's spending to grab compute is larger than anyone else's.

As a result, it has signed up almost every core compute supplier in Silicon Valley.

At the front of the line are Broadcom's $161.2 billion equipment lease and Google Cloud's $121.3 billion TPU compute (plus an additional framework of up to $150 billion).

Close behind are $120.5 billion from Musk's SpaceX and xAI (Colossus cluster), $110 billion from Amazon AWS, and after that Microsoft Azure, Nscale, Lambda, and AMD.

Comparing it with the latest financial reports of America's tech giants shows just how exaggerated this bet is.

A startup with $33.2 billion in revenue in the first three quarters already carries total off-balance-sheet commitments that exceed those of Meta ($866 billion), Microsoft ($594 billion), Nvidia ($530 billion), and Amazon ($453 billion), ranking second in the United States only to Google!

With nearly $900 billion in compute contracts on the books, Anthropic no longer has any way out to slow the pace of R&D.

To digest this enormous amount of compute and support annualized revenue of nearly $70 billion, Anthropic's only option is to hand more and more R&D work to Claude, letting it write code and run experiments at dozens of times the speed of humans!

Reference materials:

This article comes from the WeChat public account "Xinzhiyuan," author: ASI Revelations, editors: Taozi and Moses, published by 36Kr with authorization.

Source

36Kr · AI