小模型学不会大模型的复杂推理，短链更有效。
Small Models Struggle to Learn from Strong Reasoners
- 用长短结合的推理样本混合训练小模型
- 30亿参数以下模型在混合数据上表现提升显著
- 适合需要轻量化推理的部署场景
大型语言模型在复杂推理任务中表现出色，将它们的推理能力蒸馏到小型模型中具有潜力。然而，我们发现了一个新现象——小模型可学习性差距：参数量不超过30亿的小模型，并不会持续受益于长链思维（CoT）推理或来自大模型的蒸馏。相反，当使用更短、更简单的推理链进行微调时，小模型表现更好，这与其内在学习能力更匹配。为此，我们提出混合蒸馏（Mix Distillation），通过结合长链和短链推理样本，或融合大模型与小模型的推理过程，平衡推理复杂度。实验表明，相比仅使用单一类型数据，混合蒸馏能显著提升小模型的推理性能。这些发现揭示了直接从强模型蒸馏的局限性，强调了为小模型适配推理复杂度的重要性。
原文摘要 · Abstract (English)
Large language models (LLMs) excel in complex reasoning tasks, and distilling their reasoning capabilities into smaller models has shown promise. However, we uncover an interesting phenomenon, which we term the Small Model Learnability Gap: small models ($\leq$3B parameters) do not consistently benefit from long chain-of-thought (CoT) reasoning or distillation from larger models. Instead, they perform better when fine-tuned on shorter, simpler reasoning chains that better align with their intrinsic learning capacity. To address this, we propose Mix Distillation, a simple yet effective strategy that balances reasoning complexity by combining long and short CoT examples or reasoning from both larger and smaller models. Our experiments demonstrate that Mix Distillation significantly improves small model reasoning performance compared to training on either data alone. These findings highlight the limitations of direct strong model distillation and underscore the importance of adapting reasoning complexity for effective reasoning capability transfer.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。