A 10-year-old CNN beats frontier LLMs at skin cancer diagnosis

Dermatologist Magnus Lynch compared GPT-6 Astra, Claude Fable 5.1, and a fine-tuned Inception v3 on 200 biopsy-confirmed skin lesions; the CNN's top-1 diagnosis was correct in 125 cases (62%), while the two LLMs were correct in 84 and 85 cases respectively (42%). The CNN identified 37 of 50 benign nevi, while the two LLMs identified 16 and 33 respectively; each of the two LLMs had its own preferred wrong answer, with GPT-6 Astra classifying 75 lesions as basal cell carcinoma (there were actually 25). The author notes that the sample size is small, each LLM case was run only once, and Inception v3 is not the current state-of-the-art model.