实测16个模型识别蘑菇：最佳模型首猜准确率仅65%

offbeat

Even the best model gets fungus identification right just 65% of the time - talk about a false friend

Fire up that iPhone 1 emulation and party like it's 2007 11 minutes ago; Tiny11 puts Windows 11 on a diet without the PowerShell workout 1 hour ago; US tech giants warn Europe that ditching US tech giants could cost €115B 3 hours ago; The AI math isn't mathing as infrastructure spending forecast to outstrip expected revenue 6 hours ago; UK government chases AI while aging IT leaves it exposed, PAC chair warns 7 hours ago

If you see a mushroom in the forest and you're wondering if it's safe to eat or poisonous, don't ask an AI.

Polish software engineer Piotr Migdał, a founding engineer at AI analytics and cost analysis firm Quesma, explained in a Wednesday blog post why trusting the mushroom insights of leading AI models is a dubious, potentially life-threatening idea in an experiment he ran. Armed with a dataset of Danish mushrooms and additional information about safe and deadly species in Poland and abroad, he tested everything from ChatGPT to Qwen to see which gave the most reliable advice.

It likely won’t surprise you to learn that none of them are worth trusting with your life.

A total of 55 mushroom species were included in the safe and deadly datasets, which he compared against the Danish library by running 1,040 photos through 16 models, using 20 photos per species where available and topping up rarer species from the validation set. Each model was asked to identify the most likely species, and four other contenders, from each photograph.

Even the best performing model - Gemini-3.8-flash - was only correct with its first guess 65 percent of the time. It managed to correctly identify species in its top five guesses 85 percent of the time, but if you’re simply asking the model what a mushroom is and giving it a single chance, there’s a 35 percent chance it’s going to get it wrong.

Qwen3.8-27b performed worst by far, correct on the first guess 13 percent of the time, and somewhere in its five guesses 24 percent of the time.

As for what sort of mistakes the models make, most were harmless, Migdał said, but when the models got it wrong, boy did they get it wrong.

“The dangerous ones are not random,” Migdał noted. “A deadly webcap was called a chanterelle - the same mistake that kills foragers.”

The aptly named death cap mushroom was mistaken for edible 16 percent of the time, the fool’s funnel 48 percent of the time, and the fatal dapperling, often mistaken for its similar looking but safe relatives, 31 percent of the time, among others. False negatives were also noted, but there’s no harm in leaving a tasty mushroom uneaten.

As for which models made the potentially deadly misidentifications, Qwen3.8-27b led the pack there too, calling poisonous mushrooms edible 36 percent of the time, with Qwen3.8-flash following close behind at 30 percent.

Meta’s Muse-spark-1.2 led with just an eight percent false positive rate, but Migdał said that’s an anomaly due to the fact that the model simply declined to guess - the right move in a situation where a bad decision could be deadly for a user.

"I am surprised that the top generalist off-the-shelf models were able to identify most cases. Still, in this case, most is not enough, as errors could have dire consequences," Migdał told us in an email. "A single photo is often not enough to identify a species - errors are expected."

Mushroom hunting isn’t for the faint of heart

As someone who’s grown up foraging for mushrooms (it’s a popular pastime in Migdał’s Poland, and my Polish heritage has led to my family preserving the tradition in the US), I’m well versed in the fact that even well-trained humans have a tough time telling some species apart from their dangerous look-alikes.

Migdał isn't a forager himself, he explained - he'd rather accompany his friends and trust their judgment.

"I do love eating mushrooms, especially fried with nothing but butter and salt," he told The Register. "But I only join people whom I trust to know what they are doing."

That’s a big part of the reason common sense prevails in mushroom hunting: There are plenty of delicious fungi that don’t have imitators, so seek those if you’re not sure. If you trust your knowledge of mushrooms a bit more, it’s generally pretty easy to tell most safe species from their dangerous counterparts, as there are some obvious signs if you know what you’re looking at and have experience.

It’s also worth noting that death from mushroom foraging is quite rare, as some of the most deadly species out there lack lookalikes among edible varieties. You’re far more likely to end up hallucinating or with a very upset stomach if you make a mistake, but don’t take that as a license to go mushrooming if you don’t know what you’re looking at.

Most of all, “do not eat a mushroom because AI told you it is safe,” Migdał warns again in his writeup.

“AI slop … might be annoying, but it is fixable with a few prompts, or a manual edit,” he adds. “It is much better than having to prompt ‘Do I need a liver transplant?’”

Migdał's dataset is available for anyone who'd like to tweak it to see how it performs in their locality. We're guessing it won't be any more confidence inspiring than the results from Poland. ®

Fire up that iPhone 1 emulation and party like it's 2007

Browser-based fun to remember a time before Apple's annual cash grab became a thing

New motherboards, old RAM, plenty of life left in yesterday's PCs

DDR4 makes a comeback as component costs rise, while Linux offers an escape from forced upgrades

The VMware exit is a protection upgrade

PARTNER CONTENT: VergeIO says an exit is also an opportunity

Tiny11 puts Windows 11 on a diet without the PowerShell workout

New desktop app simplifies stripping out bundled software, with heavier cuts carrying security and support risks

It won't kill you, or humanity, to get off the AI upgrade treadmill

New models arrive so fast, we already have more new tech than it's possible to understand in a career

Cisco, Nvidia team up to build fatter, faster pluggable

If you thought 1.6 Tbps was fast, just wait, Cisco's USFP targets 12.8 Tbps in a single optical pluggable

TOP STORIES

Anthropic asks users to stop being mean to Claude; '123456' password linked to Denmark data breach affecting 8.8M people; Microsoft and Anthropic play invoice tennis with startup's $17,600 Claude bill; Starlink's plan to avoid orbital near-misses: Ephemeris sharing; Shai-Hulud worm makes jump to AI infrastructure with Tensorlake compromise; Dutch tax office ditches Microsoft 365 cloud for on-premises alternative

Cisco, Nvidia team up to build fatter, faster pluggable If you thought 1.6 Tbps was fast, just wait, Cisco's USFP targets 12.8 Tbps in a single optical pluggable; Claude submitted fake murder tip to Philly police during summertime testing Anthropic took more than two months to alert cops; AI systems are fully capable of carrying out nightmare attacks against infrastructure and nobody's ready Meanwhile, AI agents move robotic arms, 'make OT device operator screens lie'; Nvidia found $1B under the couch to help secure American scientific computing dominance Commitment comes as GPUzilla prepares to fortify Uncle Sam's arsenal with at least seven AI-optimized supers; TSMC taps GlobalFoundries to bolster US silicon interposer production in $2B deal Advanced packaging tech is essential to the domestic production of high-performance semiconductors used in AI datacenters

Infosec

Security Russians are posing as Signal support to launch phishing attacks PLUS: US takes down Iranian propaganda sites; Marketing company asks 'Why Do We Have Your Information?' And more!; Security Microsoft patches failed to fix on-prem SharePoint, which is now under zero-day attack PLUS: China upgrades smartphone surveillance tools; Ring eases anti-snooping stance; and more; Black Hat and DEF CON DEF CON Franklin project enlists hackers to harden critical infrastructure Voting village reports have been so successful, says Jeff Moss, that the whole of DEF CON will now be included; Security EQT buys majority share in Swiss cybersecurity biz Acronis Went at equivalent of $3.5B+ valuation for entire firm, though portion sold not specified; Malware Month Ten years since the first corp ransomware, Mikko Hyppönen sees no end in sight On the plus side, infosec's a good bet for a long, stable career

FOSS

NetBSD may have found a new role: The AI-free FOSS OS Increasing awareness and adoption, followed by increasing donations; OpenAI dots inspire open source imitators amid technical difficulties Persistent agents appeal to AI companies, but demand isn't obvious; Ubuntu 'Stonking Stingray' beta swims out: Deeper Rust 'oxidization' plus LLM speech-to-text GNOME 51 is here, and Ubuntu interim releases are getting interesting again; Revive Raskin's 3 laws of software with humane work that considers a dev's duty of care All this, and angry NeoVim users; Firefox 157: Could bold new look be just what Mozilla needs?* Yet another new version of Firefox, with a significant reskin; KDE turns 30 and someone's brought an AI-native desktop proposal Akademy talk imagines Plasma assembling itself around a personal model of each user