Top-K蒸馏会遗漏关键决策支持，导致模型误调用工具。
When Top-K Misses the Decision: Tool-Call Drift in Multi-Teacher On-Policy Distillation
- 用双教师路由机制分析蒸馏中关键支持信号的丢失问题。
- 恢复缺失的支持信号使工具误调用率从14.2%降至3.7%，但召回下降12.4点。
- 揭示了蒸馏中支持保真度与部署成本间的根本权衡，适合模型审计者阅读。
Top-K教师输出使在线蒸馏可行，但保留的教师概率质量并不能保证学生相对梯度的保真性。本文研究了路由型双教师工具使用蒸馏，在冻结的Qwen3.5-9B模型审计中，工具教师在500个工具提示中将入口标记排第一，且前32位在所有匹配对中强化该标记。而响应教师的前32位虽保留99.99%质量，却仅在0.4%的500个响应提示中包含同一标记；引入学生感知支持可恢复所有匹配对中的坐标，接近全词汇下降效果。在工具路径上，前32位虽保留强化方向，但其全词汇幅度显著衰减，尽管质量显示为1.000000。匹配修复将响应侧遗漏与行为关联：在每个监督响应位置恢复入口坐标，使全生成误调用率从14.2±2.1%降至3.7±0.5%（三种子），但调用召回下降12.4点。非工具对照组仅改变0.95点。合成输出对数梯度匹配重路由也无法复现精确修复。比较三个优化层：学生感知支持改变蒸馏坐标，损失塑形改变保留信号强度，验证调优入口偏置在不重新训练下改变推理。它们揭示了修正范围、系统访问权限与所需调用保留之间的不同权衡。Llama-3.1-8B在原生JSON协议下重现了方向性支持不对称。结果因果表明，决策关键支持遗漏是主要Qwen设置中的贡献因素之一，并强调必须同时审计支持保真度与部署成本。代码与聚合成果见https://github.com/shen-jiabin/decision-support-opd。
原文摘要 · Abstract (English)
Top-$K$ teacher logits make on-policy distillation tractable, but retained teacher mass does not certify student-relative gradient fidelity. We study routed, two-teacher tool-use distillation. In a frozen Qwen3.5-9B audit, the tool teacher ranks the entry token first on 500 tool prompts, and top-32 reinforces it in every matched pair. The response teacher's top-32 instead retains 99.99\% mass yet contains the same token on only 0.4\% of 500 response prompts; student-aware support restores the coordinate in all matched pairs and nearly matches full-vocabulary descent. On the tool route, top-32 preserves the reinforcing direction but substantially attenuates its full-vocabulary magnitude despite mass displayed as 1.000000. Matched restoration connects the response-side omission to behavior: restoring the entry coordinate at every supervised response position lowers full-generation over-calling from $14.2 \pm 2.1\%$ to $3.7 \pm 0.5\%$ across three seeds, but also lowers call recall by 12.4 points. A non-tool placebo changes over-calling by only 0.95 points. A synthetic output-logit-gradient-matched reroute also fails to reproduce exact restoration. We compare three optimization layers: student-aware support changes distilled coordinates, loss shaping changes retained-signal strength, and validation-tuned entry bias shifts inference without retraining. They expose distinct trade-offs in correction scope, system access, and required-call retention. Llama-3.1-8B reproduces the directional support asymmetry under its native JSON protocol. The results causally implicate decision-critical support omission as one contributor in the primary Qwen setting and show why both support fidelity and deployment costs must be audited. Code and aggregate artifacts are available at https://github.com/shen-jiabin/decision-support-opd.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。