←新闻详情

ARC Prize更新于 10月13日 04:12

ARC Prize 公布 DeepSeek V4.1 Flash 推理档位评测

On ARC-AGI-1, high reasoning scores 88.5% vs low's 90.5%. Scores differed on 8 tasks: high scored worse on 5 (-4.5 points) and better on 3 (+2.5 points), accoun

AI 导读

ARC Prize 公布 DeepSeek V4.1 Flash 在 ARC-AGI-1 上 high reasoning 得分 88.5%,低于 low 的 90.5%;8 个任务分数有差异,high 在 5 个任务上更差(-4.5 分)、3 个更好(+2.5 分),且输出 token 多 35% 却未带来更稳定答案。ARC-AGI-2 上 max 与 high 的每任务成本在 106 个共同完成任务中对比,max 实际贵 4.5%。ARC-AGI-3 结果将在未来几周测试完成后公布。

正文

当前显示原文,中文全文译文尚未就绪。

On ARC-AGI-1, high reasoning scores 88.5% vs low's 90.5%. Scores differed on 8 tasks: high scored worse on 5 (-4.5 points) and better on 3 (+2.5 points), accounting for the 2 percentage point drop. High used 35% more output tokens, but this didn't lead to consistently better answers.

On ARC-AGI-2, max reasoning has a slightly lower reported cost per task than high ($0.129 vs $0.133), but these averages divide recorded costs by all 120 tasks, including incomplete ones. API issues left 13 tasks incomplete with max reasoning and 8 tasks with high. Comparing the 106 complete tasks for both, max is actually 4.5% more expensive per task.

来源

ARC Prize