ARC Prize announces the evaluation results of Inkling 2 Small on ARC-AGI-1 and ARC-AGI-2…
AI summary
ARC Prize announces the evaluation results of thinkymachines' Inkling 2 Small on ARC-AGI: a score of 81.7% on ARC-AGI-2 at a cost of $0.27 per task; a score of 96.5% on ARC-AGI-1 at a cost of $0.08 per task. ARC Prize states that the model surpasses all open-weight models it has evaluated on both benchmarks, and Thinking Machines plans to release the full weights next week. ARC Prize says that evaluation on ARC-AGI-3 has higher operating costs, and results will be announced once testing is complete.
Body
@thinkymachines ARC-AGI-3 evaluations are more operationally intensive; we will share the results once testing is complete.
Full ARC-AGI-1 and ARC-AGI-2 results: https://t.co/az2H7r9gNn https://t.co/wsh6rmFA3l