用长短思维链混合训练，让大模型推理更准更简洁。
Long-Short Chain-of-Thought Mixture Supervised Fine-Tuning Eliciting Efficient Reasoning in Large Language Models
- 混合长篇与结构保留的短篇思维链数据进行微调
- 准确率提升2.3%，响应长度减少47.61%
- 适合需要高效推理的部署场景
近期研究表明，使用从大型推理模型（如DeepSeek R1）中提炼的思维链（CoT）数据对大语言模型进行监督微调（SFT），可有效将其推理能力迁移至非推理模型。然而，此类方法使模型继承教师模型的“过度思考”问题，在推理时生成冗长重复的思维链。为此，我们提出长-短思维链混合监督微调（LS-Mixture SFT），将原始长篇CoT数据与通过结构保留重写得到的短篇版本相结合。实验表明，相较于直接SFT训练的模型，采用该方法训练的模型在多个基准测试上平均准确率提升2.3%，同时响应长度显著降低约47.61%。本工作提供了一种通过监督微调赋予非推理模型推理能力的方法，且避免了源自教师模型的过度思考问题，从而实现高效推理。
原文摘要 · Abstract (English)
Recent advances in large language models have demonstrated that Supervised Fine-Tuning (SFT) with Chain-of-Thought (CoT) reasoning data distilled from large reasoning models (e.g., DeepSeek R1) can effectively transfer reasoning capabilities to non-reasoning models. However, models fine-tuned with this approach inherit the "overthinking" problem from teacher models, producing verbose and redundant reasoning chains during inference. To address this challenge, we propose Long-Short Chain-of-Thought Mixture Supervised Fine-Tuning (LS-Mixture SFT), which combines long CoT reasoning dataset with their short counterparts obtained through structure-preserved rewriting. Our experiments demonstrate that models trained using the LS-Mixture SFT method, compared to those trained with direct SFT, achieved an average accuracy improvement of 2.3% across various benchmarks while substantially reducing model response length by approximately 47.61%. This work offers an approach to endow non-reasoning models with reasoning capabilities through supervised fine-tuning while avoiding the inherent overthinking problems inherited from teacher models, thereby enabling efficient reasoning in the fine-tuned models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。