让大模型推理更短更准,避免啰嗦冗余。
ReCUT: Balancing Reasoning Length and Accuracy in LLMs via Stepwise Trails and Preference Optimization
- 分步探索生成多种推理路径,动态筛选优化。
- 推理长度减少30%-50%,准确率不降反升。
- 适合需要高效精准推理的科研与工程场景。
链式思维(CoT)提示虽显著提升大语言模型(LLMs)的推理能力,但常导致过度思考,产生冗长或重复的推理过程。现有方法通过构建多条推理链训练模型来缓解此问题,但受限于生成数据质量且易过拟合。为此,我们提出一种新方法 ReCUT:通过分步探索机制和长短路径切换采样策略,使模型逐步生成多样推理路径,并据此构建偏好对,训练两个专用模型——一个专注推理准确率,另一个侧重缩短推理长度。最终通过参数插值融合两者,得到综合性能更强的模型。在多个数学推理数据集及不同骨干模型上实验表明,ReCUT可将推理长度降低约30%-50%,同时保持或超越基线模型的准确率。代码与数据将开源于 https://github.com/NEUIR/ReCUT。
原文摘要 · Abstract (English)
Recent advances in Chain-of-Thought (CoT) prompting have substantially improved the reasoning capabilities of Large Language Models (LLMs). However, these methods often suffer from overthinking, leading to unnecessarily lengthy or redundant reasoning traces. Existing approaches attempt to mitigate this issue through curating multiple reasoning chains for training LLMs, but their effectiveness is often constrained by the quality of the generated data and prone to overfitting. To address the challenge, we propose Reasoning Compression ThroUgh Stepwise Trials (ReCUT), a novel method aimed at balancing the accuracy and length of reasoning trajectory. Specifically, ReCUT employs a stepwise exploration mechanism and a long-short switched sampling strategy, enabling LLMs to incrementally generate diverse reasoning paths. These paths are evaluated and used to construct preference pairs to train two specialized models (Gemini LLMs)-one optimized for reasoning accuracy, the other for shorter reasoning. A final integrated model is obtained by interpolating the parameters of these two models. Experimental results across multiple math reasoning datasets and backbone models demonstrate that ReCUT significantly reduces reasoning lengths by approximately 30-50%, while maintaining or improving reasoning accuracy compared to various baselines. All codes and data will be released via https://github.com/NEUIR/ReCUT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。