NYU与Amazon论文：SGUID 精选技能蒸馏，6 个技能媲美 11 倍技能库
Good paper for selecting your skill files.
NYU 与 Amazon 论文提出 SGUID，只保留训练早期和后期都持续提供有效信号的技能，用 6 个技能在数学竞赛测试上让 4 个模型中的 3 个追平或超过 30 至 71 个技能的完整技能库。按主题匹配挑选时，3 个 Qwen 模型上不足 25% 的技能能提供有效训练信号。第二轮加入 3 个新技能后，Qwen3-8B 从 64.3% 提升到 66.3%。
正文
当前显示原文，中文全文译文尚未就绪。
Good paper for selecting your skill files. Before distilling a skill bank, log which skills give a steady training signal and drop the rest.
New NYU and Amazon paper finds that distilling a few skills that keep producing a useful training signal matches or beats distilling a skill bank up to 11× larger.
Skills are short written tips, like a rule for counting cases, that a model absorbs by learning from a copy of itself that reads them. Picked by topic match, under 25% of them gave any useful signal across 3 Qwen models.
SGUID keeps only skills that help early in training and still help late. With 6 such skills, 3 of 4 models matched or beat the full bank of 30 to 71 skills on math contest tests. A 2nd round with 3 new skills lifted Qwen3-8B from 64.3% to 66.3%.