让小模型在生成时主动挑选有用推理路径,提升知识蒸馏效果
Student-in-the-Loop Chain-of-Thought Distillation via Generation-Time Selection
- 小模型在教师生成过程中实时评估推理分支,只学习可掌握的路径
- 数学推理任务上比标准蒸馏高5.9分,最优达4.7分提升
- 适合需要高效推理的小模型部署场景,如移动端或边缘设备
大型推理模型通过长链式思维(CoT)轨迹在复杂任务中表现优异,但将此类推理过程直接迁移至小型模型仍具挑战。关键难点在于,并非所有教师生成的推理路径都适合学生学习。现有方法通常采用事后过滤,在完整生成后基于启发式规则选择轨迹,但无法控制生成过程,仍可能产生超出学生学习能力的路径。为此,我们提出Gen-SSD(生成时自选择蒸馏),一种学生在环框架,实现生成时选择。学生不再被动接收完整轨迹,而是在教师采样过程中评估候选延续,仅引导可学习的推理路径扩展,并提前剪枝无益分支。在数学推理基准上的实验表明,Gen-SSD持续优于标准知识蒸馏及近期基线,相比标准KD提升约5.9分,最高达4.7分。进一步分析显示,Gen-SSD生成的推理轨迹更稳定、更易学习,凸显了在生成过程中融入监督对有效蒸馏的重要性。
原文摘要 · Abstract (English)
Large reasoning models achieve strong performance on complex tasks through long chain-of-thought (CoT) trajectories, but directly transferring such reasoning processes to smaller models remains challenging. A key difficulty is that not all teacher-generated reasoning trajectories are suitable for student learning. Existing approaches typically rely on post-hoc filtering, selecting trajectories after full generation based on heuristic criteria. However, such methods cannot control the generation process itself and may still produce reasoning paths that lie outside the student's learning capacity. To address this limitation, we propose Gen-SSD (Generation-time Self-Selection Distillation), a student-in-the-loop framework that performs generation-time selection. Instead of passively consuming complete trajectories, the student evaluates candidate continuations during the teacher's sampling process, guiding the expansion of only learnable reasoning paths and enabling early pruning of unhelpful branches. Experiments on mathematical reasoning benchmarks demonstrate that Gen-SSD consistently outperforms standard knowledge distillation and recent baselines, with improvements of around 5.9 points over Standard KD and up to 4.7 points over other baselines. Further analysis shows that Gen-SSD produces more stable and learnable reasoning trajectories, highlighting the importance of incorporating supervision during generation for effective distillation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。