ARC Prize announces evaluation results of DeepSeek V4.1 Flash 1017 on ARC-AGI
ARC Prize announces the evaluation results of DeepSeek V4.1 Flash 1017 on ARC-AGI: it scores 5.81% on the ARC-AGI-3 semi-private set, 78.1% on ARC-AGI-2, and 94.5% on ARC-AGI-1, leading among open-weight models. Compared with the original V4.1 Flash released on September 10, 2026, its ARC-AGI-3 semi-private set score rose from 2.68% to 5.81%, while its public set score fell from 11.05% to 8.97%. At maximum reasoning effort, the model's reasoning tokens consumed for the first action decreased by an average of 36% across 25 public games and by 48.6% across 55 semi-private games.
Why read this
AI score 56
ARC Prize provides the scores of DeepSeek V4.1 Flash 1017 on three ARC-AGI benchmarks along with changes in reasoning tokens, allowing comparison of the progress made against its predecessor version.
Topics
Body
@deepseek_ai @baseten - Leaderboard: https://t.co/x8U1ItNLRj - Reproduce the public results: https://t.co/vMqtds8Zet - Testing policy: https://t.co/pBoUqvXe4P - Full DeepSeek V4.1 Flash 1017 results: https://t.co/o6j71QYoOc