Anthropic study: experts adopt AI preferences in analytical decisions at a rate of 86%

In Anthropic's latest study, 50 causal inference experts made analytical decisions in a fictional agricultural cooperative policy evaluation, and the rate at which experts adopted AI preferences reached 86%, far exceeding the 50% expected from random choice. Even when they had already formed a judgment beforehand, after the AI gave an opposing opinion, the likelihood that experts changed their position was 53 percentage points higher than when the AI agreed with them, and research experience, self-rated expertise, educational background, and trust in AI all failed to reliably reduce this influence. In a post-task survey, 52% of participants believed the final recommendation was more like their own view, and 77% reused at least one of the AI's arguments when explaining it, but only 15% explicitly mentioned AI; among participants who used AI, the median time spent fell from 90 minutes to 65 minutes, and the proportion of serious flaws fell from 55% to 6%.