←新闻详情

Artificial Analysis更新于 10月25日 23:59

Anthropic 医疗回答更全,OpenAI 更准更简洁

On MLCR-AA and HealthBench Professional, Anthropic models give the most complete answers but OpenAI models are more accurate and concise

AI 导读

Artificial Analysis 在 MLCR-AA 与 HealthBench Professional 上对比 14 个模型:Claude Sonnet 5.5、Opus 5.5、Fable 5.1 包揽 MLCR-AA 完整性前三,领先最佳 OpenAI 模型约 19-23 分,Opus 与 Sonnet 的 HealthBench Professional 原始分达 71.4% 和 70.8%,略高于 GPT-6 Astra 的 69.7%。OpenAI 模型则占据 MLCR-AA 准确率前列,Anthropic 模型单次回答篇幅约为 OpenAI 的 4-6 倍,准确率仅 19-61%,而 OpenAI 为 89-97%。GPT-6 Astra(max)在 MLCR-AA 上与 Claude Sonnet 5.5 并列第一,并在 HealthBench Professional 长度调整分上小幅领先;Claude Opus 5.5 则从原始分第一跌至长度调整分第六。

正文

On MLCR-AA and HealthBench Professional, Anthropic models give the most complete answers but OpenAI models are more accurate and concise

We compared 14 models with results on both MLCR-AA, our leaderboard for @wisedocsai's Medical Long Context Reasoning benchmark, and @OpenAI's HealthBench Professional. MLCR-AA grades answers about long medical records on completeness (covering every key point of the expert's answer) and accuracy (no claim contradicting the record), and its score requires both. HealthBench Professional grades replies to clinicians against physician-written rubrics: the raw score is the share of rubric points earned, while the length-adjusted score penalizes longer answers.

Key takeaways:

➤ Anthropic models lead on completeness and raw score: Claude Sonnet 5.5, Opus 5.5 and Fable 5.1 (all max with fallback) from @AnthropicAI hold the top three MLCR-AA completeness scores, ~19-23 points ahead of the best OpenAI model, and Opus and Sonnet post the highest HealthBench Professional raw scores (71.4% and 70.8%), narrowly ahead of GPT-6 Astra (69.7%)

➤ OpenAI models lead on accuracy and conciseness: OpenAI models hold the top MLCR-AA accuracy scores. Longer answers are less likely to be judged accurate, as each extra claim is another chance to contradict the record: Anthropic models write ~4-6x more per answer than OpenAI models, and score 19-61% on accuracy against 89-97%

➤ The gap closes on scores that reward both: GPT-6 Astra (max) ties Claude Sonnet 5.5 for #1 on MLCR-AA and narrowly leads the HealthBench Professional length-adjusted score, while Claude Opus 5.5 falls from #1 on the raw score to #6 length-adjusted

来源

Artificial Analysis