Artificial Analysis Intelligence Index v5 上线，Claude…

隆重推出 Artificial Analysis Intelligence Index v5，包含 2 个新基准和 3 项升级。这是该指数首次大部分内容基于私有数据集构建 Artificial Analysis Intelligence Index 是我们对模型能力的综合衡量指标，涵盖反映当今真实使用场景的各类任务。v5 是自 1 月 v4 发布以来我们最大的一次升级。在 v5 中，该指数 60% 的内容现在基于使用私有测试集或留出答案的评估。 与 Intelligence Index v5 一同推出的还有 Artificial Analysis Progress Index v1，它在一个连续尺度上追踪模型能力随时间的变化。 Intelligence Index 新增内容： ➤ FrontierCode 2.0：Cognition 新推出的私有基准，针对真实世界编码任务，评分标准是仓库维护者是否会合并智能体的改动 ➤ Terminal-Bench-Science：70 个智能体科学研究任务，在终端环境中运行 升级内容： ➤ HLE-Diamond（取代 Humanity's Last Exam）：Humanity's Last Exam 的精炼版，共 1,000 道题，推理题和知识题各占一半 ➤ AA-Omniscience v1.1：更新后的数据集，移除了饱和题和公开集题目，新增了题目，并修订了有歧义或错误的题目 ➤ CritPt v2：CritPt v2 的私有 40 题高难度子集，由 CritPt 作者修订，并使用代码执行工具运行 保留在指数中的项目：AA-Briefcase、AutomationBench-AA、GDP.pdf 和 Terminal-Bench 4.0 从指数中移除的项目：GDPval-AA v2.1、SciCode 和 AA-LCR v1.1 主要结果： ➤ Claude Opus 5.5（max）以 58 分领先，GPT-6 Astra（max，57 分）和 GPT-6.1 Sol（max，55 分）紧随其后：Anthropic 和 OpenAI 占据前五名模型。Gemini 4 Argon（high）以 50 分排在后面 ➤ 开源权重模型的差距从 3 分扩大到 12 分：Kimi K3.1 Pro（权重尚未发布）以 46 分成为最强的开源权重模型。在已发布权重的模型中，DeepSeek V4.1 Flash 1017（max）和 GLM-5.3（max）以 32 分领先——落后领先者 26 分，且低于 Claude Haiku 5.5（max，36 分） ➤ 领先模型之间的每任务成本相差超过 13 倍：在展示的 10 个得分最高的模型中，GPT-6.1 Terra（max）最便宜，每个 Intelligence Index 任务 0.96 美元，Claude Fable 5.1（max）最贵，为 13.04 美元

查看原文

Announcing Artificial Analysis Intelligence Index v5, featuring 2 new benchmarks and 3 upgrades. For the first time, the majority of the Index is built on private datasets The Artificial Analysis Intelligence Index is our composite measure of model capabilities across tasks reflecting real-world usage today. v5 is our biggest upgrade since v4 launched in January. With v5, 60% of the Index is now based on evaluations with private test sets or held-out answers. Alongside Intelligence Index v5, we’re launching the Artificial Analysis Progress Index v1, which tracks model capability over time on one continuous scale. New to the Intelligence Index: ➤ FrontierCode 2.0: Cognition's newly-launched private benchmark of real-world coding tasks, graded on whether a repository maintainer would merge the agent's change ➤ Terminal-Bench-Science: 70 agentic scientific research tasks, run in a terminal environment Upgraded: ➤ HLE-Diamond (replacing Humanity's Last Exam): a refined 1,000-question version of Humanity's Last Exam, split evenly between reasoning and knowledge questions ➤ AA-Omniscience v1.1: An updated dataset with saturated and public set questions removed, new questions added, and ambiguous or erroneous questions revised ➤ CritPt v2: a private, 40-question hard subset of CritPt v2, revised by the CritPt authors and run with a code-execution tool Staying in the Index: AA-Briefcase, AutomationBench-AA, GDP.pdf and Terminal-Bench 4.0 Removed from the Index: GDPval-AA v2.1, SciCode, and AA-LCR v1.1 Key results: ➤ Claude Opus 5.5 (max) leads at 58, ahead of GPT-6 Astra (max, 57) and GPT-6.1 Sol (max, 55): Anthropic and OpenAI hold the top five models. Gemini 4 Argon (high) is next at 50 ➤ The open weights gap widens from 3 points to 12: Kimi K3.1 Pro (weights not yet released) is the top open weights model at 46. Among released weights, DeepSeek V4.1 Flash 1017 (max) and GLM-5.3 (max) lead at 32 - 26 points behind the leader and below Claude Haiku 5.5 (max, 36) ➤ Cost per task varies by more than 13x among the leading models: of the 10 highest-scoring models shown, GPT-6.1 Terra (max) is the cheapest at $0.96 per Intelligence Index task and Claude Fable 5.1 (max) the most expensive at $13.04