METR:如何衡量 AI 智能体的能力指标
METR 提出以智能体得分函数 s_A(x) 与人类得分函数 s_H(x) 为基础构建智能体能力指标,x 可表示花费的金钱、token 或时间。文章梳理了固定支出下的得分、实际平台期得分、固定得分下的支出、支出回报率以及支出调整后得分等指标,并指出当得分曲线回报不饱和时,固定支出得分不再适用。
ross Tasks. https://epoch.ai/data-insights/llm-inference-price-trends. Cunningham, Tom, and Manish Shetty. 2026. “An Apple-Picking Model of AI R&D.” March 13. https://tecunningham.github.io/posts/2026-03-13-apple-picking-ai.html. Cunningham, Tom, Manish Shetty, Vincent Cheng, and Nate Rush. 2026. “Expenditure Horizon: Measuring Optimization Ability, with an Application to NanoGPT.” METR, July 21. https://metr.org/blog/2026-07-21-expenditure-horizon/ appendix-d. Erol, Mehmet Hamza, Batu El, Mirac
匹配字段正文







