Chollet says Argon performs similarly to Astra on ARC-AGI-3

Oct 16

1. 23:03

   The Decoder

   Google Gemini 4 Argon scores 97.5% on ARC-AGI-3, but drops to 49.8% when the test environment is changed

   Google DeepMind's new frontier model Gemini 4 Argon scored 97.5% on ARC-AGI-3 in the Provider Adapter Harness test environment at a cost of $4,800, becoming the second lab after OpenAI to cross 95%, second only to GPT-6 Astra's 99.9%. Under the Standard Harness, which allows only simple notes, the low-reasoning setting's score dropped to 49.8%, and due to API issues it was unable to complete all 55 games. On Epoch AI's ECI index, Argon scored 165, on par with Fable 5.1, but ranked eighth on the math ECI and sixth on the software engineering index.

   Expand summary
2. 23:03

   The DecoderRelated discussion

   The Decoder joined this discussion ↗

   This post contributes to heat; it is not a formal report or confirmed fact.

3. 00:51

   François CholletSelected

   Chollet: Gemini 4 Argon performs comparably to Astra on ARC-AGI-3, Google returns to the frontier

   François Chollet stated that Gemini 4 Argon's performance on ARC-AGI-3 is close to Astra's, and Google has returned to the frontier. Data cited by ARC Prize shows that Google DeepMind's Gemini 4 Argon achieved 97.5% (provider adapter harness, $4.8K) and 49.8% (standard harness, $5.8K) on ARC-AGI-3, 91.3% at $0.32/task on ARC-AGI-2, and 98.5% at $0.16/task on ARC-AGI-1, making it the second frontier AI lab to exceed 95% on ARC-AGI-3.

   Expand summary
4. 00:22

   ARC PrizeFirst-partySelected

   ARC Prize announces evaluation results of Gemini 4 Argon on the ARC-AGI series

   ARC Prize announces the validated results of Google Gemini 4 Argon on ARC-AGI: ARC-AGI-3 reaches 97.5% (provider adapter harness, cost $4.8K), and 49.8% under the standard harness (cost $5.8K); ARC-AGI-2 is 91.3%, at $0.32 per task; ARC-AGI-1 is 98.5%, at $0.16 per task. ARC Prize states that Google DeepMind is the second frontier AI lab to exceed 95% on ARC-AGI-3, and Argon's 97.5% is second only to GPT-6 Astra's 99.9%. The standard harness test failed to complete all 55 games due to API issues, and the high reasoning tier completed only 19 games so no score was published; costs are calculated at the current discounted price of $2 per million input tokens and $10 per million output tokens.

   Expand summary
5. 00:22

   arcprizeFirst-partySelected

   English translation is being prepared.

6. 00:22

   arcprizeFirst-partySelected

   English translation is being prepared.