用技能库指导大模型推理，让错误更易被发现。
Skill-Conditioned Gated Self-Distillation for LLM Reasoning
- 从经验技能库中检索错误案例，构建多教师评分池
- 通过验证器判断教师意见正误，生成精准监督信号
- 适合提升数学推理能力，尤其在弱可信信息下表现好
在策略自蒸馏（SD）中，利用教师端的特权信息（PI）将稀疏的验证结果转化为密集的词元级监督。现有方法通常假设PI可信，如参考答案或成功轨迹。本文探讨是否可由经验积累的技能库提供PI——这类技能虽紧凑可复用，但可能无关或误导。为此提出技能条件门控自蒸馏（SGSD），将基于技能的蒸馏视为教师假设验证而非无条件模仿。SGSD检索技能-错误对，构建多教师池，让所有技能条件教师评分同一学生输出。验证器判断每名教师立场：支持成功或抑制失败则给予正向监督，反之则反转。鲁棒的门控目标进一步蒸馏有信息量的师生分歧，抑制不确定或极端信号。在多个数学推理基准测试中，SGSD持续优于GRPO，且在较弱的PI假设下仍媲美答案条件的OPSD。例如，在Qwen3-1.7B上，于AIME24、AIME25和HMMT25平均提升6.2%（相比GRPO）和1.7%（相比OPSD）。
原文摘要 · Abstract (English)
On-policy self-distillation (SD) improves LLM reasoning by using teacher-side privileged information (PI) to turn sparse verifier outcomes into dense token-level supervision. Existing methods usually assume trusted PI, such as reference answers or successful traces. We ask whether PI can instead come from an experience-derived skill bank, where retrieved skills are compact and reusable but may also be irrelevant or misleading. We propose Skill-Conditioned Gated Self-Distillation (SGSD), which formulates skill-based SD as teacher hypothesis validation rather than unconditional imitation. SGSD retrieves skill-mistake pairs, constructs a multi-teacher pool, and lets all skill-conditioned teachers score the same plain-prompt student rollout. The verifier validates each teacher's polarity: supporting a success or suppressing a failure gives positive supervision, while the opposite stance is reversed. A robust gated objective then distills informative teacher-student disagreements while suppressing uncertain or extreme signals. Experiments on multiple mathematical reasoning benchmarks show that SGSD consistently improves over GRPO and remains competitive with answer-conditioned OPSD under a weaker PI assumption. For example, on Qwen3-1.7B, SGSD outperforms GRPO by 6.2% and OPSD by 1.7% on average on AIME24, AIME25, and HMMT25.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。