Harvard study: Paper reviews written by Claude Opus 4.8 outperform human reviews on all metrics
A Harvard study used Claude Opus 4.8 to generate reviews for 1,124 papers and conducted a blind comparison against 134 expert reviews from rapidPeer. The results showed that the AI reviews outperformed the best expert reviews on all metrics tested by the authors. GPT-5.5 chose the AI review in all 132 decided pairs, and 3 former journal editors chose the AI review 87% of the time; erroneous statements about the papers accounted for 5.5% of AI reviews versus 15.9% of human reviews. Based on this, the authors suggest that teams relying on expert review can have AI complete a full first-round review, with humans making the final decision.
Why read this
AI score 54
This comparison provides concrete gaps between AI and human reviewers in error rates and blind-review win rates, which can inform the boundaries of AI participation in peer review.
Topics
Body
Massive. New Harvard study finds that AI-written paper reviews now beat the best human peer reviews on every measure the authors tested.
Peer review is slow and often misses errors. Earlier studies concluded AI should only assist human reviewers, not replace them.
They used 134 reviews from rapidPeer, a Harvard service that pays expert reviewers well. Claude Opus 4.8 reviewed the same 1124 papers, and GPT-5.5 plus 3 former journal editors judged the pairs blind.
GPT-5.5 picked the AI review in all 132 decided pairs, and the editors picked it 87% of the time. Only 5.5% of the AI's claims about the paper were wrong, versus 15.9% for the human reviews.
Teams that depend on expert critique can now let AI do the first full review pass, and keep a human for the final decision.