ARC Prize 评测 Claude Fable 5.2:ARC-AGI-3 标准 harness 得分 81.2%
At max reasoning with the standard harness, 28 of 55 semi-private games reached the 12-hour runtime limit, capping how far testing could progress. Max scored 65
ARC Prize 公布 Anthropic Claude Fable 5.2 在 ARC-AGI 系列基准上的成绩:ARC-AGI-3 在标准 harness 下得分 81.2%、成本 21.9K 美元,在 provider adapter harness 下为 97.7%、18.6K 美元;ARC-AGI-2 为 93.8%、每任务 0.98 美元,ARC-AGI-1 为 98.5%、每任务 0.36 美元。在 max 推理档下,55 个半私有游戏中有 28 个触及 12 小时运行时限,得分 65.7%,低于 xhigh 档的 81.2%,ARC Prize 认为若无时限 max 档会更高。ARC Prize 还说明,已发布的 Fable 5.2 在 medium 和 xhigh 推理档上的调优与早前评测的 checkpoint 不同,low、high、max 档未变。
推荐理由
AI 评分 56
ARC Prize 给出 Fable 5.2 在 ARC-AGI 三个基准上的分数与成本,并说明 max 推理档受 12 小时时限影响。
主题
正文
At max reasoning with the standard harness, 28 of 55 semi-private games reached the 12-hour runtime limit, capping how far testing could progress. Max scored 65.7% on ARC-AGI-3 versus 81.2% at xhigh; without the limit, we expect max would have scored higher.
Also note that the released Fable 5.2 uses different tuning at medium and xhigh reasoning effort than the earlier checkpoint we evaluated. Low, high, and max are unchanged.