arXiv:2601.09088cs.LGcs.CL2026-01被引 14

用分布对齐提升小模型长链推理能力,效果超越更大模型。

Distribution-Aligned Sequence Distillation for Superior Long-CoT Reasoning

  • 通过分布对齐改进序列蒸馏,让小模型更好学习大模型的推理分布。
  • 仅用44.8万样本即达到同类开源模型最优,比多数方法少一个数量级。
  • 适合想用小模型实现强推理的开发者,尤其关注高效训练与开源可用性。

本文提出DASD-4B-Thinking,一个轻量但高性能的全开源推理模型,在数学、科学推理和代码生成等挑战性基准上表现领先,甚至超越部分更大规模模型。我们重新审视当前主流的序列级蒸馏范式——基于教师生成响应的监督微调(SFT)。尽管该方法在实践中效率高、性能好,但其本质仍局限于SFT视角,侧重设计启发式数据过滤规则,却忽视了蒸馏的核心:使学生模型完整学习教师的输出分布以继承泛化能力。我们识别出三大问题:教师序列分布表征不足;师生能力不匹配;教师强制训练与自回归推理间的暴露偏差。这些问题反映出蒸馏过程中缺乏显式的师生互动。为此,我们提出一系列创新,构建了增强型序列级蒸馏训练流程。值得注意的是,DASD-4B-Thinking仅使用44.8万条训练样本便取得竞争力结果,远低于现有大多数开源方案。为支持社区研究,我们公开发布模型与训练数据集。

原文摘要 · Abstract (English)

In this report, we introduce DASD-4B-Thinking, a lightweight yet highly capable, fully open-source reasoning model. It achieves SOTA performance among open-source models of comparable scale across challenging benchmarks in mathematics, scientific reasoning, and code generation -- even outperforming several larger models. We begin by critically reexamining a widely adopted distillation paradigm in the community: SFT on teacher-generated responses, also known as sequence-level distillation. Although a series of recent works following this scheme have demonstrated remarkable efficiency and strong empirical performance, they are primarily grounded in the SFT perspective. Consequently, these approaches focus predominantly on designing heuristic rules for SFT data filtering, while largely overlooking the core principle of distillation itself -- enabling the student model to learn the teacher's full output distribution so as to inherit its generalization capability. Specifically, we identify three critical limitations in current practice: i) Inadequate representation of the teacher's sequence-level distribution; ii) Misalignment between the teacher's output distribution and the student's learning capacity; and iii) Exposure bias arising from teacher-forced training versus autoregressive inference. In summary, these shortcomings reflect a systemic absence of explicit teacher-student interaction throughout the distillation process, leaving the essence of distillation underexploited. To address these issues, we propose several methodological innovations that collectively form an enhanced sequence-level distillation training pipeline. Remarkably, DASD-4B-Thinking obtains competitive results using only 448K training samples -- an order of magnitude fewer than those employed by most existing open-source efforts. To support community research, we publicly release our models and the training dataset.

模型蒸馏长链推理开源模型轻量化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。