Claude Sonnet 5.5's results on the ARC-AGI benchmark announced
ARC Prize announces Anthropic's Claude Sonnet 5.5's results on each generation of the ARC-AGI benchmark: ARC-AGI-3 is 72.4% under the provider adapter harness at a cost of 8.2K USD, and 24.3% under the standard harness at a cost of 7.4K USD; ARC-AGI-2 is 83.9% at 0.55 USD per task; ARC-AGI-1 is 96.5% at 0.14 USD per task. ARC Prize says Sonnet 5.5's 72.4% on ARC-AGI-3 is noteworthy.
Why read this
SelectedAI score 68
ARC Prize announces Claude Sonnet 5.5's scores and per-task costs on three generations of the ARC-AGI benchmark, allowing comparison of performance differences under different harnesses.
Topics
Body
Claude Sonnet 5.5 from @AnthropicAI on ARC-AGI (Verified):
- ARC-AGI-3: 72.4%, $8.2K (provider adapter harness), 24.3%, $7.4K (standard harness) - ARC-AGI-2: 83.9%, $0.55/task - ARC-AGI-1: 96.5%, $0.14/task
Sonnet 5.5 is notable for its 72.4% score on ARC-AGI-3. https://t.co/FBKS6gf96l