METR Report: Anthropic's Model Improvement Rate Is About Twice the 2024 to 2025 Pace
After spending 11 weeks reviewing Anthropic's internal R&D data and interviewing its core researchers, METR released a report stating that Anthropic's current rate of model improvement is about twice the 2024 to 2025 pace, and that recursive self-improvement has moved from theory to something measurable. METR estimates that about 80% of the current acceleration comes from Anthropic's own use of AI, and ruled out faster growth in training compute as the cause; the report also concludes that over the next 12 months, algorithms, compute, and data will not constitute bottlenecks. The report notes that Anthropic's public risk report in August underestimated the actual pace of progress, that Anthropic withheld most of the evidence METR considered most persuasive on the grounds that it could help competitors, and that it canceled midway an examination of how compute is allocated between capability and safety.
Why read this
SelectedAI score 80
Using 11 weeks of internal data, METR turned recursive self-improvement from theory into measurable numbers and attributed the sources of the acceleration.
Topics
View original postSince August, a team within METR has worked with internal Anthropic information to assess whether AI itself is accelerating AI development. We are now publishing a report summarizing the results of this risk assessment.
View original
Since August, a team within METR has worked with internal Anthropic information to assess whether AI itself is accelerating AI development. We are now publishing a report summarizing the results of this risk assessment.
Body
So much revelation from METR's latest report.
Recursive self-improvement (RSI) at Anthropic has moved from theory to solid measurement, and the early loop already runs at about 2x.
After they spent 11 weeks going through Anthropic’s internal R&D data and interviewing its key researchers.
> Anthropic is now improving its models at about twice its 2024 to 2025 pace.
> Claude itself is the main engine. METR’s best guess puts about 80% of current acceleration on Anthropic’s own AI use, and it rules out faster growth in training compute as the cause.
> Anthropic’s own public August risk report understated how fast things were moving. METR reached that conclusion in a separate review that compared the public report with the private version and other nonpublic data.
> METR saw nothing likely to slow this loop within 12 months, with no bottleneck in algorithms, compute or data. Redacted evidence also hints that acceleration may already exceed 2x and keep climbing.
> Staff self-report roughly 12x productivity gains from Model 4, and researchers see “signs of life” for AI doing key training work alone. One called Mythos Preview the first model useful for autonomous experiments.
> Anthropic redacted most of the evidence METR found most telling, saying it could help rivals go faster. It also canceled midway a planned check of how its compute splits between capabilities and safety.