New preprint tests 15 models on peer review: implanted-error detection rate 53% to 96%, human reviewers average 27%
A new preprint tested the ability of 15 models to detect implanted errors in clinical trial papers, with scores ranging from 53% to 96%, while untrained human reviewers averaged only 27%. The results still held on 58 new trials published in 2026, showing that the models were not merely recalling old papers seen during training; Claude Opus 5.5 scored highest at 99%. Human reviewers could detect errors such as text-table contradictions—the type that most affects trial meaning—in only 7% of cases, and each review took 5 to 8.5 hours, whereas AI took 1.4 to 16.3 minutes.
Why read this
AI score 73
The original text presents a comparison of 15 models' detection rates on implanted errors versus human peer review, which can be used to understand the real-world performance of AI-assisted peer review.
Topics
View original postLots of debate this week on whether journals should ban AI in peer review. We actually tested this in our new preprint, and the results were pretty clear to me. For context, I've reviewed for most of the big journals and evaluated 1,000 human reviews as an editor. I love to read and evaluate science. TLDR: From now on I won't do any peer review without AI. For clinical trials, I'd even call human-only review dangerous 🧵
View original
Lots of debate this week on whether journals should ban AI in peer review. We actually tested this in our new preprint, and the results were pretty clear to me. For context, I've reviewed for most of the big journals and evaluated 1,000 human reviews as an editor. I love to read and evaluate science. TLDR: From now on I won't do any peer review without AI. For clinical trials, I'd even call human-only review dangerous 🧵
Body
> AI now catches far more errors in clinical trial papers than human peer reviewers: all 15 models tested scored 53% to 96% on planted errors, while untrained human reviewers averaged just 27%.
> The result held on 58 new trials published in 2026, so the models weren’t simply recalling old papers they had seen in training; Claude Opus 5.5 scored highest at 99%.
> Humans miss the errors that most change a trial’s meaning, catching text-table contradictions only 7% of the time, and they take 5 to 8.5 hours per review against AI’s 1.4 to 16.3 minutes.