通过模型插值生成推理数据谱,提升小模型的链式思维蒸馏效果。
MI-Distillation: Selecting from Model-Interpolated Instruct-Reasoning Data Spectrum for Chain-of-Thought Distillation

- 用模型插值构建连续推理数据谱,融合长短思维路径优势。
- 在多个推理基准上优于强基线,小模型性能显著提升。
- 适合追求高效推理蒸馏的小模型研究者使用。
大型推理模型(LRM)在复杂问题上展现出强大的长链式思维(Long CoT)能力。然而,将此类推理轨迹蒸馏到小型学生模型仍具挑战:直接使用长链监督效果有限,甚至不如简洁短链(Short CoT)推理。本文从梯度角度分析发现,长链推理产生的梯度幅度更大、更新方向更集中,且随着学生模型容量增加,该现象更明显。这表明有效长链蒸馏需平衡推理信息密度与学生模型分布对齐。为此,我们提出模型插值蒸馏(MI-Distillation),通过模型插值构建连续的指令-推理数据谱。为从中选择合适路径,引入可学习序列惊奇度评分(SeqLSS),偏好对学生有信息量且可学习的推理路径。在多个推理基准上的实验表明,MI-Distillation 在小模型链式思维蒸馏中持续优于强基线。
原文摘要 · Abstract (English)
Recent advances in large reasoning models (LRMs) have shown strong performance on complex problems through long chain-of-thought (Long CoT) reasoning. However, distilling such trajectories into smaller student models remains challenging: direct Long CoT supervision often provides limited gains and can be less effective than concise Short CoT rationales. In this work, we investigate this phenomenon from a gradient-centric perspective. Our analysis shows that Long CoT induces larger gradient magnitudes and more concentrated update directions than Short CoT, with this effect becoming more pronounced as student model capacity increases. These findings suggest that effective Long CoT distillation requires balancing the reasoning information density of reasoning trajectories with their distributional alignment to the student model. Motivated by this insight, we propose \textbf{M}odel \textbf{I}nterporlation \textbf{Distillation} (\textbf{MI-Distillation}), a framework that constructs a continuous Instruct-Reasoning data spectrum through model interpolation. To select suitable trajectories from this spectrum, we further introduce \textbf{Seq}uential \textbf{L}earnable \textbf{S}urprisal \textbf{S}core (\textbf{SeqLSS}), which favors reasoning paths that are both informative and learnable for the student. Extensive experiments on reasoning benchmarks show that MI-Distillation consistently improves small model CoT distillation over strong Long CoT baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。