ARC Prize publishes Meta Muse Spark Open's evaluation results on ARC-AGI
ARC Prize published the validation results of Meta's Muse Spark Open on ARC-AGI: ARC-AGI-1 at 89.0% with $0.29 per task, ARC-AGI-2 at 57.5% with $0.73 per task, and ARC-AGI-3 at 0.1% under the standard harness with $3.9K. ARC Prize states that its v1/2 scores are close to those of Haiku 5.5 and GPT-6 Luna, but the cost is about 9 times that of the latter; the model is priced on OpenRouter at $1.25 per million input tokens and $4.25 per million output tokens. As the reasoning tier increases, ARC-AGI-2 rises from 29.4% at medium (averaging 55k tokens/task) to 57.2% at xhigh (180k tokens/task).
Why read this
SelectedAI score 62
ARC Prize gives Meta's new model's scores on three versions of ARC-AGI along with per-task cost, directly comparable with competitors scoring the same.
Topics
Body
Muse Spark Open from @AIatMeta on ARC-AGI (Verified):
- ARC-AGI-3: 0.4%, $3.0K (provider adapter harness), 0.1%, $3.9K (standard harness) - ARC-AGI-2: 57.5%, $0.73/task - ARC-AGI-1: 89.0%, $0.29/task
Its scores on v1/2 are similar to Haiku 5.5 & GPT-6 Luna, but ~9x the cost. https://t.co/PtKFobKDc4