METR 与 7 位经济学家合著论文《递归自我改进的经济学》

Our Work

Risk Assessment; About; Donate; Careers

We (Parker and Tom) recently coauthored a paper, “The Economics of Recursive Self-Improvement”, with 7 other economists. The paper walks through a series of simple models of how AI may accelerate AI R&D, and we thought it’s worth highlighting some context and takeaways:

We care about Recursive Self-Improvement (RSI) because we want to forecast capabilities. METR’s priority is to assess risk from frontier AI development, and one input is how capable AI systems will be in the future. Capabilities have been growing rapidly over the past 5 years, and we want to know whether to expect an acceleration.1; The term RSI has been used with very different definitions. Unfortunately a lot of confusion has been caused by different definitions of RSI. Everyone agrees that RSI refers to feedback from model capabilities to model improvements, but some have said that RSI occurs when there’s any feedback (Karpathy, Patel, Musk, LessWrong), while others reserve it for when the feedback is strong enough to cause super-exponential growth (Lambert) or fully autonomous growth (Favaro & Clark). We decided not to use the term RSI in a technical sense, to avoid confusion. Instead we focus on the strength of feedback effects, and whether they are sufficiently strong for “self-sustaining acceleration.” (We have a longer survey of definitions here); The effect on capabilities acceleration depends on the strength of feedback effects. The model gives a simple way of quantifying the strength of overall feedback effects through decomposing into individual effects. The most uncertain relationship is how an increase in model capabilities would increase the rate of algorithmic progress; We can’t rule out a substantial acceleration. We discuss a variety of reasons why there could be an acceleration in capabilities that fizzles out: bottlenecks on data, training compute, inference compute, or experiments; algorithmic-specific capabilities; and R&D-specific capabilities. However, we do not think the evidence for any of these is overwhelming; we cannot rule out an extended and rapid acceleration in capabilities; There is more data relevant to RSI that the labs could be releasing. Over the past 6 months labs have released a lot of useful data about the impact of AI on AI R&D (Mythos model card; GPT-5.6 model card; Favaro & Clark), but there are many more facts they could release that would be useful. The paper gives one specific “wish list” for future releases; What next? The paper has a calibration, suggesting estimates for parameters, but it is very loose and meant to be a first draft. We hope to keep iterating on our quantitative model to give a more operationally useful model of RSI

We are also interested in reasons why capabilities might decelerate, e.g. our paper on a slowdown in the growth of training compute. ↩

Want to contribute to this work? METR is hiring: View open roles

Featured research

METR researches, develops and runs cutting-edge tests of AI capabilities, including broad autonomous capabilities and the ability of AI systems to conduct AI R&D.

A survey of 349 technical workers finds a median 1.4–2x self-reported change in value of work due to AI tools, expected to grow over time, though there are reasons to be skeptical of the magnitude.

Early Work on Monitorability Evaluations

We show preliminary results on a prototype evaluation that tests monitors' ability to catch AI agents doing side tasks, and AI agents' ability to bypass this monitoring.

How Does Time Horizon Vary Across Domains?

We build on our time-horizon work and analyze 9 benchmarks for scientific reasoning, math, robotics, computer use, and self-driving in terms of time-horizon trends; we observe generally similar rates of improvement to the 7-month doubling time in our original time-horizon work.