Artificial Analysis 发布 Intelligence Index v5,60% 评测基于私有数据集
Announcing Artificial Analysis Intelligence Index v5, featuring 2 new benchmarks and 3 upgrades. For the first time, the majority of the Index is built on priva
Artificial Analysis 发布 Intelligence Index v5,新增 FrontierCode 2.0 和 Terminal-Bench-Science 两个基准,升级 HLE-Diamond、AA-Omniscience v1.1 和 CritPt v2,移除 GDPval-AA v2.1、SciCode 和 AA-LCR v1.1,60% 的指数改为基于私有测试集或留出答案的评测。同时推出 Progress Index v1,用统一尺度追踪模型能力随时间的变化。结果显示 Claude Opus 5.5(max)以 58 分居首,GPT-6 Astra(max,57)和 GPT-6.1 Sol(max,55)紧随,Gemini 4 Argon(high)为 50 分;开源权重模型与领先者的差距从 3 分扩大到 12 分,已发布权重中 DeepSeek V4.1 Flash 1017(max)和 GLM-5.3(max)以 32 分领先。前十名模型单任务成本相差超过 13 倍,GPT-6.1 Terra(max)最低为 $0.96,Claude Fable 5.1(max)最高为 $13.04。
推荐理由
精选AI 评分 65
原文列出 v5 指数新增与升级的基准及移除项,并给出头部模型分数与单任务成本差距,可据此比较各模型能力与开销。
主题
正文
Announcing Artificial Analysis Intelligence Index v5, featuring 2 new benchmarks and 3 upgrades. For the first time, the majority of the Index is built on private datasets
The Artificial Analysis Intelligence Index is our composite measure of model capabilities across tasks reflecting real-world usage today. v5 is our biggest upgrade since v4 launched in January. With v5, 60% of the Index is now based on evaluations with private test sets or held-out answers.
Alongside Intelligence Index v5, we’re launching the Artificial Analysis Progress Index v1, which tracks model capability over time on one continuous scale.
New to the Intelligence Index:
➤ FrontierCode 2.0: Cognition's newly-launched private benchmark of real-world coding tasks, graded on whether a repository maintainer would merge the agent's change
➤ Terminal-Bench-Science: 70 agentic scientific research tasks, run in a terminal environment
Upgraded:
➤ HLE-Diamond (replacing Humanity's Last Exam): a refined 1,000-question version of Humanity's Last Exam, split evenly between reasoning and knowledge questions
➤ AA-Omniscience v1.1: An updated dataset with saturated and public set questions removed, new questions added, and ambiguous or erroneous questions revised
➤ CritPt v2: a private, 40-question hard subset of CritPt v2, revised by the CritPt authors and run with a code-execution tool
Staying in the Index: AA-Briefcase, AutomationBench-AA, GDP.pdf and Terminal-Bench 4.0
Removed from the Index: GDPval-AA v2.1, SciCode, and AA-LCR v1.1
Key results:
➤ Claude Opus 5.5 (max) leads at 58, ahead of GPT-6 Astra (max, 57) and GPT-6.1 Sol (max, 55): Anthropic and OpenAI hold the top five models. Gemini 4 Argon (high) is next at 50
➤ The open weights gap widens from 3 points to 12: Kimi K3.1 Pro (weights not yet released) is the top open weights model at 46. Among released weights, DeepSeek V4.1 Flash 1017 (max) and GLM-5.3 (max) lead at 32 - 26 points behind the leader and below Claude Haiku 5.5 (max, 36)
➤ Cost per task varies by more than 13x among the leading models: of the 10 highest-scoring models shown, GPT-6.1 Terra (max) is the cheapest at $0.96 per Intelligence Index task and Claude Fable 5.1 (max) the most expensive at $13.04