METR Report: Anthropic's Model Capability Growth Rate Is About 1.9 Times the Previous Rate, with 80% of the Increase Coming from AI

On November 5, the third-party evaluation organization METR released an investigation report on Anthropic, stating that the benchmark capability progress this company achieved over the past year roughly corresponds to the progress made in the previous two years.

From August 12 to October 29, this organization spent about 11 weeks going inside Anthropic to examine its model results, training compute, R&D efficiency, and other materials, and conducted interviews with researchers.

Their analysis used Anthropic's version of the composite capability metric AECI. The data shows that since October 2025, the rate of progress of large models on this metric has been about 1.9 times the previous rate, with a 95% confidence interval of 1.5–2.8 times.

Since October 7, 2025, this score has jumped from about 15 points/year to about 29 points/year, covering in one year the ground previously covered in about two years, roughly equivalent in magnitude to the span from Claude 2.1 to Sonnet 4.5. Anthropic's own risk report had previously assumed that Mythos Preview was a one-time jump after which the trend would return to its original trajectory; however, new models have now appeared, and the new slope continues.

Looking across the entire industry, the acceleration of frontier models is only 1.4 times.

In the interviews, a recurring phenomenon was that researchers can finally try more ideas.

In the past, an experiment might take one or two days to implement, and the time and compute costs were insurmountable, so many ideas were shelved before they could be validated. Today, researchers can hand multiple ideas to Agents to see which ones are worth pursuing further.

One researcher described their work as now mainly limited by ideas and compute, with code no longer the bottleneck. Others feel that using Claude is like having a postdoc and a graduate student advancing a project together. Experiments and implementation have become cheap, making researchers more willing to try things, and many small experiments previously deemed not worth the effort now have a chance to become new research directions.

METR's report also reveals some internal progress that was previously invisible to the outside world. Their analysis incorporated three Anthropic models that have not yet been made public, referred to as Model 2, Model 3, and Model 4. They completed training between July and October of this year.

The strongest currently is this Model 4 (some guess it is Claude 6), which has already exceeded 90% on many common benchmarks. Anthropic employees made internal subjective estimates of it in October, voting that on average it improved their work efficiency by about 12 times.

These models have not been released, and may not even become public products, but their existence reminds the outside world that looking only at launch events and public leaderboards may not fully capture what is happening inside major AI companies. From about 1 time in early 2025, to about 2.5 times for Opus 4.5, about 5 times for Mythos 5, and a jump to 12 times for Model 4, the exponential fit is 3.9 times per year.

METR estimates that as of October this year, about 80% of the current acceleration in AI research is estimated to come from AI. There is another 20% from non-AI factors, the details of which Anthropic did not allow to be disclosed.

There is still a gap before achieving complete recursive self-improvement (RSI), in which AI independently develops the next generation of models. Anthropic's own researchers believe that in the most important part of model training, AI has not yet made highly important and fully autonomous contributions, and the main gains currently still come from AI-assisted human R&D.

However, Anthropic researchers have already described AI's substantial autonomous contributions as "signs of life," and there is quantitative analysis statistically tallying the share of tasks completed by AI versus humans in each stage, but the data was deleted by Anthropic.

As long as stronger models can help researchers develop the next generation faster, the feedback process may continue to strengthen. How far the RSI path can go depends on the stages that have not yet been automated, and on whether resources such as compute can keep up.

Regarding predictions for the near future, METR does not see bottlenecks in the next 12 months that would clearly block rapid progress, but also frankly acknowledges that as information increased, they changed their judgment multiple times, and their confidence is currently limited.

In addition, although Anthropic allowed the third-party organization METR to enter and investigate internally, the 100-page report currently provided has undergone substantial redactions by them. Most of the most persuasive evidence was deleted by Anthropic itself.

A companion review released the same day also presented the two sides' disagreement over risk wording: Anthropic had described a certain leading indicator as showing a rate of progress "less than two times," while METR calculated about 1.97 times based on its internal data and believed the more accurate description should be "about two times." The twofold acceleration involves Anthropic's early warning threshold.

METR believes that the possibility that the relevant threshold had already been crossed as of July 15 cannot be ruled out, but most relevant researchers still lean toward it not having been crossed at that time. Both sides also agree that the direct catastrophic risk of the relevant models at that time was low.

Reference content:

This article comes from the WeChat public account "Machine Heart" (ID: almosthuman2014), author: Machine Heart, which follows RSI, editor: Zenan, published by 36Kr with authorization.