Arena 分析 20869 条记录：前沿编程智能体三大差异
Where do users actually feel the difference with a frontier coding agent?
Arena 基于 Agent Arena: Code 的 20,869 条 trace，对比 Anthropic 与 OpenAI 五款顶级模型与其他智能体的用户批评信号，发现前沿模型在三个维度差异最明显。失败修复循环的抱怨少 6.9 个百分点，忽略项目上下文少 1.9 个百分点；设计问题少 6.4 个百分点，产出可用结果的问题少 5.8 个百分点。指令遵循、任务完成、破坏已有代码的抱怨分别少 5.6、4.9、4.5 个百分点。
正文
Where do users actually feel the difference with a frontier coding agent?
Across 20,869 Agent Arena traces, we compared how users criticized the five top models (all from @AnthropicAI & @OpenAI) on Agent Arena: Code, versus how they criticized other agents.
Here are 3 areas where frontier models make the most difference:
1. Less getting stuck: Among sessions with complaints, failed repair loops come up 6.9 percentage points less often, and project context gets ignored less often too (-1.9 pp). You spend less time steering the agent out of dead ends.
2. Better taste and code that works: They are better at design (-6.4 pp) and at producing working results (-5.8 pp). What you get back is more likely to be what you wanted.
3. More diligence: They follow instructions better (-5.6 pp), finish the work more often (-4.9 pp) and break existing work less (-4.5 pp). There's less to clean up afterwards.
Curious for a deeper dive into Agent Arena's praise vs complaint signal? See the article below in thread.