AI Trends

ARC Prize's hands-on test of Inception AI's Mercury 3 Preview…

ARC PrizeUpdated Nov 17 23:22

AI summary

ARC Prize tested Inception AI's Mercury 3 Preview on the ARC-AGI benchmark: it scored 61.8% on ARC-AGI-2 at $0.06 per task, 88.5% on ARC-AGI-1 at $0.03 per task, and only 0.2% on ARC-AGI-3. With the instant reasoning tier enabled, the model averaged 981 tokens/second output on the public ARC-AGI-2 dataset, with a peak of 1,743 tokens/second in a single call (3,001 tokens output in 1.72 seconds); with reasoning enabled, it averaged 500-600 tokens/second. ARC Prize says this is the first diffusion model it has evaluated.

Why read this

AI score 55

ARC Prize's hands-on test of Mercury 3 Preview provides scores and throughput data across reasoning tiers, allowing comparison of its speed versus accuracy trade-offs.

Topics

Tags

Body

@_inception_ai With reasoning enabled, Mercury 3 Preview averaged 500-600 tokens per second: https://t.co/1bHNxkjFZh

Source

ARC Prize