arXiv:2509.08814cs.LGcs.AI2025-09被引 5

用多教师推理融合提升小模型推理能力,效果超越大模型。

Merge-of-Thought Distillation

  • 通过交替训练与合并多个教师的推理路径,融合不同教师优势。
  • 仅用200个样本,小模型在数学竞赛题上超越Deepseek-R1等大模型。
  • 适合需要高效压缩复杂推理能力的场景,如部署在移动端的智能助手。

长链式思维(CoT)模型的高效推理蒸馏常受限于单一教师假设,尽管实际存在多个候选教师和日益丰富的CoT语料库。我们重新审视教师选择问题,发现不同学生所需的最优教师不同,同一学生在不同数据集上最佳教师也不同。为统一多教师推理能力并解决监督信号冲突,我们提出轻量级框架Merge-of-Thought Distillation(MoT),其在教师特定微调分支与学生变体权重空间合并之间交替进行。在竞赛数学基准测试中,仅使用约200个CoT样本,将Qwen3-14B作为学生应用MoT后,性能超越Deepseek-R1、Qwen3-32B及OpenAI-O1等强模型。此外,MoT持续优于单教师蒸馏,提升跨领域通用推理能力,减少灾难性遗忘,并对分布偏移及同级别教师具有鲁棒性。最终,我们验证了MoT具备共识性推理能力,消除教师特异性归纳偏置与教师间冲突,反复强化共识推理特征的学习。这些结果表明,MoT是高效从多样教师中蒸馏长链推理能力至紧凑学生的简单而有效路径。

原文摘要 · Abstract (English)

Efficient reasoning distillation for long chain-of-thought (CoT) models is increasingly constrained by the assumption of a single oracle teacher, despite the practical availability of multiple candidate teachers and growing CoT corpora. We revisit teacher selection and observe that different students have different "best teachers," and even for the same student, the best teacher can vary across datasets. Therefore, to unify multiple teachers' reasoning abilities into a student to overcome conflicts among various teachers' supervision, we propose Merge-of-Thought Distillation (MoT), a lightweight framework that alternates between teacher-specific supervised fine-tuning branches and weight-space merging of the resulting student variants. On competition math benchmarks, using only about 200 CoT samples, applying MoT to a Qwen3-14B student surpasses strong models including Deepseek-R1, Qwen3-32B, and OpenAI-O1, demonstrating substantial gains. Besides, MoT consistently outperforms the best single-teacher distillation, improves general reasoning beyond mathematics while reducing catastrophic forgetting, and shows robustness to distribution-shifted and peer-level teachers. Finally, we have demonstrated MoT possesses consensus CoT by eliminating teacher-specific inductive biases and inter-teacher conflicts while repeatedly reinforcing the learning of consensus reasoning features. These results position MoT as a simple, effective route to efficiently distilling long CoT capabilities from diverse teachers into compact students.

推理蒸馏多教师学习链式思维模型压缩

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。