ARC Prize evaluates Claude Fable 5.2: 81.2% score on the ARC-AGI-3 standard harness
ARC Prize announced Anthropic Claude Fable 5.2's results on the ARC-AGI series of benchmarks: ARC-AGI-3 scored 81.2% under the standard harness at a cost of $21.9K, and 97.7% at $18.6K under the provider adapter harness; ARC-AGI-2 scored 93.8% at $0.98 per task, and ARC-AGI-1 scored 98.5% at $0.36 per task. At the max reasoning tier, 28 of 55 semi-private games hit the 12-hour runtime limit, scoring 65.7%, lower than the 81.2% of the xhigh tier; ARC Prize believes the max tier would have scored higher without the time limit. ARC Prize also noted that the released Fable 5.2's tuning on the medium and xhigh reasoning tiers differs from the checkpoint used in earlier evaluations, while the low, high, and max tiers are unchanged.
Why read this
AI score 56
ARC Prize reports Fable 5.2's scores and costs on the three ARC-AGI benchmarks, and notes that the max reasoning tier was affected by the 12-hour time limit.
Topics
Body
At max reasoning with the standard harness, 28 of 55 semi-private games reached the 12-hour runtime limit, capping how far testing could progress. Max scored 65.7% on ARC-AGI-3 versus 81.2% at xhigh; without the limit, we expect max would have scored higher.
Also note that the released Fable 5.2 uses different tuning at medium and xhigh reasoning effort than the earlier checkpoint we evaluated. Low, high, and max are unchanged.