通过分枝合并蒸馏，让小模型在数学等任务上超越大模型。
TinyR1-32B-Preview: Boosting Accuracy with Branch-Merge Distillation
- 先分领域训练小模型，再合并实现跨域知识融合。
- 在数学、编程和科学任务上分别提升5.5、4.4、2.9分。
- 适合追求高性能小型化模型的研究者与开发者。
减小大型语言模型（LLM）规模的同时保持性能，已成为重要研究方向。现有方法如模型蒸馏和迁移学习常难以兼顾高精度。为此，我们提出分枝-合并蒸馏方法：第一阶段（分枝）通过领域特定的监督微调，从大教师模型中选择性地提取知识到专用学生模型；第二阶段（合并）将多个学生模型融合，促进跨领域知识迁移并提升泛化能力。我们以DeepSeek-R1为教师模型，DeepSeek-R1-Distill-Qwen-32B为学生模型进行验证。最终生成的合并模型TinyR1-32B-Preview，在多个基准测试中表现优异，数学 (+5.5)、编程 (+4.4) 和科学 (+2.9) 任务均优于其对应模型，且在AIME 2024上接近DeepSeek-R1的性能。该方法为构建更小、高效、高性能的LLM提供了可扩展解决方案，显著降低计算成本与训练时间。
原文摘要 · Abstract (English)
The challenge of reducing the size of Large Language Models (LLMs) while maintaining their performance has gained significant attention. However, existing methods, such as model distillation and transfer learning, often fail to achieve high accuracy. To address this limitation, we introduce the Branch-Merge distillation approach, which enhances model compression through two phases: (1) the Branch Phase, where knowledge from a large teacher model is \textit{selectively distilled} into specialized student models via domain-specific supervised fine-tuning (SFT); And (2) the Merge Phase, where these student models are merged to enable cross-domain knowledge transfer and improve generalization. We validate our distillation approach using DeepSeek-R1 as the teacher and DeepSeek-R1-Distill-Qwen-32B as the student. The resulting merged model, TinyR1-32B-Preview, outperforms its counterpart DeepSeek-R1-Distill-Qwen-32B across multiple benchmarks, including Mathematics (+5.5 points), Coding (+4.4 points) and Science (+2.9 points), while achieving near-equal performance to DeepSeek-R1 on AIME 2024. The Branch-Merge distillation approach provides a scalable solution for creating smaller, high-performing LLMs with reduced computational cost and time.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。