论文发现 AI 文本检测器在模型换代后漏检率骤升
This paper finds that AI-text detectors trained on a vendor's older models caught over 99% of rewrites before a generation change and only 3.8% after it.
一篇论文发现，基于某厂商旧模型训练的 AI 文本检测器在模型换代前能识别超过 99% 的改写文本，换代后仅能识别 3.8%。该研究来自东京都立大学，指出检测器漏检比例与改写所用 LLM 版本高度相关，例如对 GPT-5 改写的摘要识别率为 93.5%，而对另一个新模型改写的摘要漏检率达 79.8%。作者建议依赖此类检测器筛查科学论文的人，在每次模型发布后重新测试。
查看原帖Pangram, the AI-text detector, missed 79.8% of scientific abstracts rewritten by Meta's Muse-Glimmer, while flagging just 1 of 5,000 human abstracts. In a new paper from Tokyo Metropolitan University, reseaerchers find the share of AI-rewritten abstracts that Pangram misses depends strongly on the LLM version Shows that it caught 93.5% of GPT-5 rewrites…
展开完整原文
Pangram, the AI-text detector, missed 79.8% of scientific abstracts rewritten by Meta's Muse-Glimmer, while flagging just 1 of 5,000 human abstracts. In a new paper from Tokyo Metropolitan University, reseaerchers find the share of AI-rewritten abstracts that Pangram misses depends strongly on the LLM version Shows that it caught 93.5% of GPT-5 rewrites but missed 79.8% from another new model. Its miss rate depended mostly on which model did the rewriting.
正文
当前显示原文，中文全文译文尚未就绪。
This paper finds that AI-text detectors trained on a vendor's older models caught over 99% of rewrites before a generation change and only 3.8% after it.
Detectors that screen scientific papers for AI writing can stop working when a new LLM generation arrives, so anyone relying on them should re-test them with every model release.