解决学生模型多样性不足问题，用更低成本实现教师模型的思维多样性迁移。
Influence-Directed Distillation: Solving the Diversity Bottleneck in Sampled-Token On-Policy Distillation
- 基于教师采样词的概率，动态区分更新方向，保留有益熵增，替换有害熵减。
- 在推理任务上显著提升 pass@$k$，同时保持 pass@1 不下降，成本更低。
- 适合追求高效且高多样性知识迁移的模型压缩与蒸馏场景。
采样词在线策略蒸馏（OPD）通过学生生成的词元高效迁移教师能力，仅需采样词的教师概率。然而常出现多样性蒸馏失败：学生 pass@1 提升但 pass@$k$ 停滞，无法继承教师的多样性。为此，我们提出一阶局部熵影响（First-Order Local Entropy Influence），一个带符号的一阶代理指标，将每次更新的熵效应解耦为教师-学生对数概率差与学生局部概率结构，并实证关联熵收缩与负影响位置。受此启发，我们提出影响导向自适应在线策略蒸馏（IDA-OPD）：不依赖昂贵的全词汇表前向KL目标，而是保留熵扩张更新，用发散自适应优势缩减替换熵收缩更新，仅需教师采样词的对数概率。在以推理为导向的蒸馏实验中，IDA-OPD 稳定提升 pass@$k$，成功继承教师多样性，达到最强教师信息方法的性能，且成本更低，同时广泛保持原生 OPD 的 pass@1，全程无需全词汇表教师信息。
原文摘要 · Abstract (English)
Sampled-token on-policy distillation (OPD) efficiently transfers capabilities from teacher to student using student-generated tokens, requiring teacher probabilities only for sampled tokens. Yet it frequently suffers from diversity distillation failure: the student's pass@1 improves while its pass@$k$ plateaus, failing to inherit the teacher's diversity. To explain this, we introduce First-Order Local Entropy Influence, a signed first-order proxy that decouples each update's entropy effect into the teacher--student log-probability gap and the student's local probability structure, and empirically links entropy contraction to negative-influence positions. Motivated by this, we propose Influence-Directed Adaptive On-Policy Distillation (IDA-OPD): rather than relying on costly full-vocabulary Forward-KL objectives, it preserves entropy-expanding updates while replacing entropy-contracting ones with divergence-adaptive advantage shrinkage, using only the teacher's sampled-token log-probability. Experiments on reasoning-oriented distillation show IDA-OPD consistently improves pass@$k$, inheriting the teacher's diversity through distillation, matches the strongest teacher-informed methods at strictly lower cost, and broadly maintains vanilla OPD's pass@1, all without full-vocabulary teacher information.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。