CMU 与 Microsoft 论文发布 CRBugBench：最佳 AI 代码审查智能体 top 5 评论命中 71.9% 已知 bug
New Carnegie Mellon and Microsoft paper finds that the best AI code reviewer caught 71.9% of known bugs in its top 5 comments, still too few to remove human rev
卡内基梅隆大学与 Microsoft 的新论文提出 CRBugBench，包含 1800 个代码审查任务，每个 pull request 都引入了一个经人工确认并修复的 bug，智能体在沙箱中审查原始改动，由 AI 评审判断其 top 5 评论是否指出该 bug。表现最好的 claude-opus-5 在 top 5 评论中命中 71.9% 的 bug，5 个月前的 opus-4.6 为 47.7%；在某公司私有仓库上性能最多下降 25.5%，这类 bug 更依赖项目上下文。作者建议将审查智能体与人工审查配合使用，并向其提供仓库自身知识，尤其是在大型或上下文密集的改动上。
正文
当前显示原文，中文全文译文尚未就绪。
New Carnegie Mellon and Microsoft paper finds that the best AI code reviewer caught 71.9% of known bugs in its top 5 comments, still too few to remove human reviewers.
Agents now write code faster than people can review it, tempting teams to let AI review it too. Older review benchmarks scored AI against human comments, which can be nitpicks or miss real bugs.
They built CRBugBench, 1,800 code review tasks where each pull request introduced a bug that humans confirmed and fixed. The agent reviews the original change in a sandbox, and AI judges check whether its top findings name that bug.
The best agent, claude-opus-5, caught 71.9% of bugs in its top 5 comments, up from 47.7% for opus-4.6 5 months earlier. Performance dropped up to 25.5% on 1 company's private repos, where bugs more often need project context.
Use review agents alongside human reviewers, and give them your repo's own knowledge, especially on large or context-heavy changes.