Artificial Analysis v5 and the Benchmarker's Dilemma
Nathan Lambert reviews the newly released Intelligence Index v5 from Artificial Analysis, which weights knowledge work, coding, science, and general each at 25%, and presents new model rankings. He notes that 8 closed-source models in the rankings lead the best open-source model, Kimi K3.1 Pro, and argues that the new index leans more toward measuring where frontier models are heading, potentially underestimating the role of open-source models in the economy. The article also discusses the pros and cons of shifting to private benchmarks, and how the business model of benchmark organizations charging model vendors to verify results affects credibility.
#English text is being prepared.#Evaluations / benchmarks#Industry news
