ARC Prize evaluates Inkling 2 Small: 81.7% on ARC-AGI-2, $0.27 per task
ARC Prize has published validation results for Thinky Machines' Inkling 2 Small on ARC-AGI: 81.7% on ARC-AGI-2 at $0.27 per task, and 96.5% on ARC-AGI-1 at $0.08 per task. ARC Prize says it surpasses all previously evaluated open-weight models on both benchmarks, and compared with the first-generation Inkling Small released on July 30, its score on the ARC-AGI-2 semi-private set rose from 40.14% to 81.67%. Thinking Machines plans to release the full weights next week, and ARC-AGI-3 results will be announced after testing is complete.
Why read this
AI score 71
ARC Prize's independent evaluation reports Inkling 2 Small's scores on ARC-AGI-1 and ARC-AGI-2 and its per-task cost, allowing direct comparison with the previous generation and other open-weight models.
Topics
Body
Inkling 2 Small from @thinkymachines on ARC-AGI (Verified):
- ARC-AGI-2: 81.7%, $0.27/task - ARC-AGI-1: 96.5%, $0.08/task
It outperforms every open-weight model we've evaluated on both ARC-AGI-1 & ARC-AGI-2.
Thinking Machines plans to release its full weights next week. https://t.co/jrwbBEqeTf