arXiv:2503.20641cs.CL2025-03被引 67

用模型合并技术让大模型高效推理,减少冗余思考。

Unlocking Efficient Long-to-Short LLM Reasoning with Model Merging

  • 通过融合不同模型能力,实现快速思考与深度推理结合。
  • 推理长度平均缩短55%,性能不降反升。
  • 适合需要高效精准推理的应用场景,如智能客服、自动写作。

大型语言模型从系统1到系统2的推理转变,在处理复杂任务时展现出显著进步,但往往伴随效率下降,存在过度思考问题。长到短(L2S)推理旨在平衡推理深度与实际效率。现有方法如监督微调、强化学习和提示工程成本高或不稳定。模型合并则提供了一种低成本且稳健的替代方案,将系统1的快速反应与系统2的严谨推理相结合。本文对模型合并用于L2S推理进行了全面实证研究,探索了基于任务向量、SVD及激活信息的多种合并方法。实验表明,模型合并可使平均响应长度减少高达55%,同时保持或提升基线性能。在1.5B/7B/14B/32B多个规模模型上验证了模型规模与合并效果之间的强相关性。此外,研究还发现合并后模型具备自评与纠错能力,并能根据任务复杂度动态调整响应长度。结果表明,模型合并是实现高效、可靠L2S推理的有效范式,为解决过度思考问题提供了实用方案。

原文摘要 · Abstract (English)

The transition from System 1 to System 2 reasoning in large language models (LLMs) has marked significant advancements in handling complex tasks through deliberate, iterative thinking. However, this progress often comes at the cost of efficiency, as models tend to overthink, generating redundant reasoning steps without proportional improvements in output quality. Long-to-Short (L2S) reasoning has emerged as a promising solution to this challenge, aiming to balance reasoning depth with practical efficiency. While existing approaches, such as supervised fine-tuning (SFT), reinforcement learning (RL), and prompt engineering, have shown potential, they are either computationally expensive or unstable. Model merging, on the other hand, offers a cost-effective and robust alternative by integrating the quick-thinking capabilities of System 1 models with the methodical reasoning of System 2 models. In this work, we present a comprehensive empirical study on model merging for L2S reasoning, exploring diverse methodologies, including task-vector-based, SVD-based, and activation-informed merging. Our experiments reveal that model merging can reduce average response length by up to 55% while preserving or even improving baseline performance. We also identify a strong correlation between model scale and merging efficacy with extensive evaluations on 1.5B/7B/14B/32B models. Furthermore, we investigate the merged model's ability to self-critique and self-correct, as well as its adaptive response length based on task complexity. Our findings highlight model merging as a highly efficient and effective paradigm for L2S reasoning, offering a practical solution to the overthinking problem while maintaining the robustness of System 2 reasoning. This work can be found on Github https://github.com/hahahawu/Long-to-Short-via-Model-Merging.

模型合并推理优化大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。