ARC Prize publishes Muse Spark Open reasoning scaling test results on ARC-AGI
ARC Prize published the test results of Meta's Muse Spark Open on the public ARC-AGI tasks: ARC-AGI-2 improves with reasoning scale, with medium averaging 55k tokens per task for 29.4%, high at 150k tokens for 51.0%, and xhigh at 180k tokens for 57.2%. ARC-AGI-1 already reached 92.8% at high and rose to 94.0% at xhigh. The model scores 57.5% on ARC-AGI-2 at $0.73 per task, and 89.0% on ARC-AGI-1 at $0.29 per task; its scores are close to those of Haiku 5.5 and GPT-6 Luna, but its cost is about 9 times theirs.
Why read this
AI score 53
The original text gives Muse Spark Open's scores and token consumption on ARC-AGI-2 as the reasoning budget increases, which can be used to judge the trade-off between cost and benefit.
Topics
Body
@AIatMeta @baseten On public tasks, Muse Spark Open gained most on ARC-AGI-2 as reasoning scaled: medium averaged 55k tokens/task for 29.4%, high 150k for 51.0%, and xhigh 180k for 57.2%. ARC-AGI-1 was already 92.8% at high and rose to 94.0% at xhigh. https://t.co/SEZU8mNtMv