Understand this story
On October 16, 2026, three authors from the Department of Computing at Imperial College London published the paper "Retrieved but Not Read: Understanding Why Search-Augmented LLMs Fail to Use Available Evidence," investigating why search-augmented LLMs frequently cite web pages they never actually opened. The study tested six open-weight configurations—GPT-5.6 Luna, GLM-5.3-Flash, gpt-oss-120b, gpt-oss-20b, gemma-4-31b, and qwen3-8b—using 225 facts that changed after February 16, 2026. The results showed that models often answer based solely on search snippets without opening the source pages; under the same set of search results, the proportion of turns in which no page was opened ranged from 2.6% to 76%, and when Luna used its built-in search, 53.6% of turns relied only on snippets. When answering from snippets alone, outdated answers accounted for 6.4% and misleading citations for 6.4%; when pages were opened and cited, these two figures dropped to 1.3% and 2.3%, respectively. The study also found that placing the correct page directly into the prompt significantly reduced errors, but some models still favored old information even when shown new facts. Currently, this research has been reported by a single source, with no subsequent developments yet.
AI-generated from public reports
Latest development · Report
Imperial College London paper: search-augmented LLMs often cite web pages they never openedDevelopments
1 confirmed developments
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Oct 16
Unite.AI
Imperial College London paper: search-augmented LLMs often cite web pages they never opened
Three authors from Imperial College London have published the paper "Retrieved but Not Read: Understanding Why Search-Augmented LLMs Fail to Use Available Evidence," examining why search-augmented LLMs frequently cite web pages they never actually open. The study tested GPT-5.6 Luna and six open-weight configurations including GLM-5.3-Flash, gpt-oss-120b, gpt-oss-20b, gemma-4-31b, and qwen3-8b using 225 facts that changed after February 16, 2026, finding that models often answer based solely on search snippets without opening the source pages; under the same set of search results, the proportion of cases where no page was opened ranged from 2.6% to 76%, and when Luna used its built-in search, 53.6% of turns relied only on snippets. When answering from snippets alone, outdated answers accounted for 6.4% and misleading citations for 6.4%, whereas when pages were opened and cited, these fell to 1.3% and 2.3% respectively; placing the correct page directly into the prompt significantly reduced errors, but some models still favored old information even when shown the new facts.