AI Trends

ARC Prize announces evaluation results of Mercury 3 Preview on ARC-AGI

ARC PrizeUpdated Nov 17 23:22

AI summary

ARC Prize announced the evaluation results of Inception's Mercury 3 Preview on ARC-AGI: ARC-AGI-1 score 88.5%, $0.03 per task; ARC-AGI-2 score 61.8%, $0.06 per task; ARC-AGI-3 score 0.2%, $299 per task (provider adapter) or 0.1%, $333 (standard). This is the first diffusion model evaluated by ARC Prize; under instant reasoning effort it averages 981 output tokens/second on the ARC-AGI-2 public dataset, with a peak of 1,743 tokens/second for a single call; with reasoning enabled it averages 500-600 tokens/second. ARC Prize also provided links to the leaderboard, reproduction method, and testing policy.

Why read this

SelectedAI score 60

ARC Prize announced the scores and inference costs of Inception's Mercury 3 Preview on the three ARC-AGI benchmarks, and provided a reproducible entry point.

Topics

Tags

Body

@_inception_ai - Leaderboard: https://t.co/x8U1ItNLRj - Reproduce the public results: https://t.co/vMqtds8Zet - Testing policy: https://t.co/pBoUqvXe4P - Full Mercury 3 Preview results: https://t.co/F4A7ENPd6Q

Source

ARC Prize