Artificial Analysis releases Intelligence Index v5, with 60% of evaluations based on private datasets
Artificial Analysis releases Intelligence Index v5, adding two new benchmarks, FrontierCode 2.0 and Terminal-Bench-Science, upgrading HLE-Diamond, AA-Omniscience v1.1 and CritPt v2, removing GDPval-AA v2.1, SciCode and AA-LCR v1.1, and changing 60% of the index to evaluations based on private test sets or held-out answers. It also launches Progress Index v1, which uses a unified scale to track changes in model capabilities over time. The results show that Claude Opus 5.5 (max) ranks first with 58 points, followed by GPT-6 Astra (max, 57) and GPT-6.1 Sol (max, 55), while Gemini 4 Argon (high) scores 50; the gap between open-weight models and the leaders has widened from 3 points to 12 points, and among released weights, DeepSeek V4.1 Flash 1017 (max) and GLM-5.3 (max) lead with 32 points. The per-task cost of the top ten models differs by more than 13 times, with GPT-6.1 Terra (max) lowest at $0.96 and Claude Fable 5.1 (max) highest at $13.04.
Why read this
SelectedAI score 65
The original text lists the benchmarks added and upgraded in the v5 index as well as the items removed, and provides the scores of leading models and the per-task cost gap, which can be used to compare the capabilities and costs of each model.
Topics
Body
Announcing Artificial Analysis Intelligence Index v5, featuring 2 new benchmarks and 3 upgrades. For the first time, the majority of the Index is built on private datasets
The Artificial Analysis Intelligence Index is our composite measure of model capabilities across tasks reflecting real-world usage today. v5 is our biggest upgrade since v4 launched in January. With v5, 60% of the Index is now based on evaluations with private test sets or held-out answers.
Alongside Intelligence Index v5, we’re launching the Artificial Analysis Progress Index v1, which tracks model capability over time on one continuous scale.
New to the Intelligence Index:
➤ FrontierCode 2.0: Cognition's newly-launched private benchmark of real-world coding tasks, graded on whether a repository maintainer would merge the agent's change
➤ Terminal-Bench-Science: 70 agentic scientific research tasks, run in a terminal environment
Upgraded:
➤ HLE-Diamond (replacing Humanity's Last Exam): a refined 1,000-question version of Humanity's Last Exam, split evenly between reasoning and knowledge questions
➤ AA-Omniscience v1.1: An updated dataset with saturated and public set questions removed, new questions added, and ambiguous or erroneous questions revised
➤ CritPt v2: a private, 40-question hard subset of CritPt v2, revised by the CritPt authors and run with a code-execution tool
Staying in the Index: AA-Briefcase, AutomationBench-AA, GDP.pdf and Terminal-Bench 4.0
Removed from the Index: GDPval-AA v2.1, SciCode, and AA-LCR v1.1
Key results:
➤ Claude Opus 5.5 (max) leads at 58, ahead of GPT-6 Astra (max, 57) and GPT-6.1 Sol (max, 55): Anthropic and OpenAI hold the top five models. Gemini 4 Argon (high) is next at 50
➤ The open weights gap widens from 3 points to 12: Kimi K3.1 Pro (weights not yet released) is the top open weights model at 46. Among released weights, DeepSeek V4.1 Flash 1017 (max) and GLM-5.3 (max) lead at 32 - 26 points behind the leader and below Claude Haiku 5.5 (max, 36)
➤ Cost per task varies by more than 13x among the leading models: of the 10 highest-scoring models shown, GPT-6.1 Terra (max) is the cheapest at $0.96 per Intelligence Index task and Claude Fable 5.1 (max) the most expensive at $13.04