ARC Prize Releases ARC-AGI Verification Results for Gemini 4 Hydrogen
Gemini 4 Hydrogen scored 92.5% and 98.5% on ARC-AGI-2 and ARC-AGI-1, respectively, at a per-task cost of $1.26 and $0.18. At the high reasoning level, it matched or outperformed Gemini 3.8 Flash while using approximately 52% fewer reasoning tokens. ARC Prize was unable to complete ARC-AGI-3 testing due to API issues involving multi-action function calls, MALFORMED_FUNCTION_CALL errors, and related problems.
Why read this
AI score 51
The results present accuracy, cost per task, and reasoning token consumption, while also documenting API compatibility issues encountered during ARC-AGI-3 testing.
Topics
Body
Gemini 4 Hydrogen from @GoogleDeepMind on ARC-AGI (Verified):
- ARC-AGI-2: 92.5%, $1.26/task - ARC-AGI-1: 98.5%, $0.18/task
Gemini 4 Hydrogen matched or exceeded Gemini 3.8 Flash's scores on both benchmarks at high reasoning while using about 52% fewer reasoning tokens. https://t.co/WSYNtbPstx
AI Trends