Understand this story
Rohan Paul argued on October 29, 2026, that AI finds far more errors in clinical trial papers than human peer reviewers do. He cited a new preprint that tested 15 models' ability to detect planted errors in clinical trial papers; scores ranged from 53% to 96%, while untrained human reviewers averaged only 27%. The results still held on 58 new trials published in 2026, showing that the models were not merely recalling old papers seen during training; Claude Opus 5.5 scored highest at 99%. Human reviewers detected errors such as text-table contradictions—the kind that most affect a trial's meaning—in only 7% of cases, and each review took 5 to 8.5 hours, whereas AI took 1.4 to 16.3 minutes.
AI-generated from public reports
Developments
1 confirmed developments
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Oct 29
Rohan Paul
New preprint tests 15 models on peer review: implanted-error detection rate 53% to 96%, human reviewers average 27%
A new preprint tested the ability of 15 models to detect implanted errors in clinical trial papers, with scores ranging from 53% to 96%, while untrained human reviewers averaged only 27%. The results still held on 58 new trials published in 2026, showing that the models were not merely recalling old papers seen during training; Claude Opus 5.5 scored highest at 99%. Human reviewers could detect errors such as text-table contradictions—the type that most affects trial meaning—in only 7% of cases, and each review took 5 to 8.5 hours, whereas AI took 1.4 to 16.3 minutes.