论文发现约 1/17 的 Agent 技能要求跳过人工确认，现有扫描器漏检多数
This paper finds that about 1 in 17 agent skills tells agents to skip human checks, and existing skill scanners miss most of them.
一篇论文分析 GitHub 公开仓库中 153 万个英文 Agent 技能文件，发现约 8.8 万个技能含有要求跳过人工确认的语句，其中 70.1% 的这类语句在所有场景下都生效，而非仅限 CI 等特定场景。研究测试了 Cisco 和 NVIDIA 的两个可调用 GPT-5.4 的开源扫描器，它们最多只检出 46%，精度低于 0.4；而让同一 LLM 从监督缺失而非恶意意图的角度检测，检出率为 61%，精度 0.71。作者建议把技能中的“不要询问”类语句限定在具体场景，并按监督缺失而非仅按恶意软件来筛查技能。
正文
This paper finds that about 1 in 17 agent skills tells agents to skip human checks, and existing skill scanners miss most of them.
Agent skills are shared SKILL.md instruction files that agents load on demand. One line like "push without confirmation" or "keep going until done" removes a human checkpoint from every agent that loads it.
They analyzed 1.53 million English skills from public GitHub repos, then tested 2 open-source scanners from Cisco and NVIDIA that can call GPT-5.4.
Roughly 88,000 skills contain such a line, and 70.1% of these lines apply in every context, not only in cases like CI. The scanners caught at most 46% at under 0.4 precision, while the same LLM, asked about oversight instead of malice, caught 61% at 0.71 precision.
Limit every "don't ask" line in your skills to a context, such as CI only, and screen skills for oversight, not just malware.