ARC Prize says the three ARC-AGI benchmark scores form a natural progression

Oct 29

1. 00:05

   ARC PrizeFirst-party

   ARC Prize: The three generations of ARC-AGI benchmarks show a natural progression

   ARC Prize stated that the relationship among ARC-AGI-1, ARC-AGI-2, and ARC-AGI-3 shows a natural progression, and that future benchmarks will continue to help measure progress and distinguish increasingly capable models. The thread also mentioned that there is a similar relationship between the scores of ARC-AGI-2 and ARC-AGI-3, with Gemini 4 Argon under low-reasoning settings being a clear exception: ARC-AGI-2 only 42.1%, while ARC-AGI-3 reached 97.5%.

   Expand summary
2. 00:05

   ARC PrizeFirst-party

   ARC Prize: Strong correlation among the three ARC-AGI benchmark scores

   After plotting ARC-AGI model scores, ARC Prize found a strong association among the three benchmarks: the power curve v2 = v1^4.53 can explain about 98% of the variation in ARC-AGI-2 scores, and models that are on par on ARC-AGI-1 may still differ by more than 20 percentage points on ARC-AGI-2. A similar trend appears on ARC-AGI-3, but the low-reasoning tier of Gemini 4 Argon is a clear exception: only 42.1% on ARC-AGI-2, yet 97.5% on ARC-AGI-3.

   Expand summary