AI Trends

Trending story

Gemini 4 Hydrogen Matches or Outperforms Gemini 3.8 Flash on Two ARC-AGI Benchmarks

1 reports1 story updatesUpdated Nov 21 00:57

Understand this story

AI overview

ARC Prize released validation results for Gemini 4 Hydrogen: the model scored 92.5% on ARC-AGI-2 and 98.5% on ARC-AGI-1, respectively, matching or outperforming Gemini 3.8 Flash at the high reasoning level while using approximately 52% fewer reasoning tokens; the per-task costs for the two tests were $1.26 and $0.18, respectively. ARC Prize also said that ARC-AGI-3 testing could not be completed due to API issues involving multi-action function calling, MALFORMED_FUNCTION_CALL, and other problems. There are currently no other evidence-backed subsequent developments.

AI-generated from public reports

Latest development · Report

ARC Prize Releases ARC-AGI Verification Results for Gemini 4 Hydrogen

Developments

1 confirmed developments

  1. Gemini 4 Hydrogen Matches or Outperforms Gemini 3.8 Flash on Two ARC-AGI Benchmarks

    Source:ARC Prize

Reports and discussion timeline

Browse reports and discussion by time. Their evidence levels remain distinct.

Nov 21

  1. ARC PrizeFirst-party

    ARC Prize Releases ARC-AGI Verification Results for Gemini 4 Hydrogen

    Gemini 4 Hydrogen scored 92.5% and 98.5% on ARC-AGI-2 and ARC-AGI-1, respectively, at a per-task cost of $1.26 and $0.18. At the high reasoning level, it matched or outperformed Gemini 3.8 Flash while using approximately 52% fewer reasoning tokens. ARC Prize was unable to complete ARC-AGI-3 testing due to API issues involving multi-action function calls, MALFORMED_FUNCTION_CALL errors, and related problems.

Related stories

  1. Related storiesGemini 4 Hydrogen Matches or Surpasses Its Predecessor’s Scores on Two ARC-AGI Benchmarks
  2. Related storiesGemini 4 Hydrogen Achieves Verified ARC-AGI Scores
  3. Related storiesGemini 4 Neon Begins Rolling Out on the Gemini and Antigravity Platforms
  4. Related storiesGoogle launches Gemini 4 Hydrogen on Antigravity