Imperial College London paper: search-augmented LLMs often cite web pages they never opened
Three authors from Imperial College London have published the paper "Retrieved but Not Read: Understanding Why Search-Augmented LLMs Fail to Use Available Evidence," examining why search-augmented LLMs frequently cite web pages they never actually open. The study tested GPT-5.6 Luna and six open-weight configurations including GLM-5.3-Flash, gpt-oss-120b, gpt-oss-20b, gemma-4-31b, and qwen3-8b using 225 facts that changed after February 16, 2026, finding that models often answer based solely on search snippets without opening the source pages; under the same set of search results, the proportion of cases where no page was opened ranged from 2.6% to 76%, and when Luna used its built-in search, 53.6% of turns relied only on snippets. When answering from snippets alone, outdated answers accounted for 6.4% and misleading citations for 6.4%, whereas when pages were opened and cited, these fell to 1.3% and 2.3% respectively; placing the correct page directly into the prompt significantly reduced errors, but some models still favored old information even when shown the new facts.
Three authors from Imperial College London have published the paper "Retrieved but Not Read: Understanding Why Search-Augmented LLMs Fail to Use Available Evidence," examining why search-augmented LLMs frequently cite web pages they never actually open. The study tested GPT-5.6 Luna and six open-weight configurations including GLM-5.3-Flash, gpt-oss-120b, gpt-oss-20b, gemma-4-31b, and qwen3-8b using 225 facts that changed after February 16, 2026, finding that models often answer based solely on
Matched fieldsBody
