Artificial Analysis releases Intelligence Index v5 and Progress…
Artificial Analysis released Intelligence Index v5 and the new Progress Index in San Francisco. Intelligence Index v5 adds two benchmarks and three upgrades, increases the weighting of agentic coding and science, and for the first time makes most of the index based on private test sets or held-out answers. The Progress Index uses item response theory to estimate a general capability score for each model across dozens of evaluations measured over time, including evaluations that have been retired or excluded from the Intelligence Index. It builds on prior work by Epoch AI Research and Anthropic, placing models on a single capability scale. According to the official statement, the Intelligence Index remains the flagship metric for choosing which model to use right now, while the Progress Index is better suited to depicting long-term trends.
Why read this
AI score 51
The original text explains the benchmarks and weighting changes newly added to the new version of the index, as well as the division of roles between the Progress Index and the Intelligence Index in terms of use.
Topics
Body
On Tuesday, we brought together partners and friends in San Francisco to launch Artificial Analysis Intelligence Index v5 and our new Progress Index
Micah Hill-Smith (@_micah_h) introduced Intelligence Index v5, including two new benchmarks and three upgrades, with greater weighting on agentic coding and science. For the first time, the majority of the Index is based on private test sets or held-out answers.
We also introduced Artificial Analysis Progress Index, which uses item response theory to estimate a general capability score for each model across dozens of evaluations measured over time, including those retired or excluded from the Intelligence Index. It builds on prior work from @EpochAIResearch and @AnthropicAI to place models on a single capability scale. Intelligence Index remains the flagship metric for choosing which current model to use for a use case today; Progress Index is the better tool for plotting trends over time.
George Cameron (@grmcameron) led a discussion with Silas (@silasalberti) from @cognition, Steven (@StevenDillmann) from @Stanford and Terminal-Bench-Science, and our own Declan Jackson about building evaluations as models become more capable.
The evening also covered our work on 3D modeling, personality, cyber defense, custom enterprise evaluations and robotics. Thank you to our speakers, partners and everyone who joined us!
Explore Intelligence Index v5: https://t.co/YI6GbwBUFD Explore Progress Index: https://t.co/Bdfnpw5mhw