让大模型推理更简洁,减少重复思考,长度降50%仅损失2%准确率
Optimizing Length Compression in Large Reasoning Models
- 提出简洁性与充分性新原则,针对性优化冗余推理
- 通过双奖励机制压缩推理链,长度减少约50%,准确率仅降2%
- 适合追求高效推理的部署场景,尤其适合资源受限环境
大型推理模型虽表现优异,但常产生冗长无用的推理过程。我们识别出核心问题是‘无效思考’——模型在得出正确答案后仍反复验证。为此,提出两个细粒度原则:简洁性(消除冗余)和充分性(保留关键步骤)。基于此,提出后训练方法LC-R1,采用组相对策略优化(GRPO),结合长度奖励与压缩奖励,专门去除无效推理部分。多基准测试显示,LC-R1实现约50%的序列长度压缩,准确率仅下降约2%,在帕累托前沿上达到高压缩优先的优良平衡。分析验证了其鲁棒性,为开发更高效的大模型提供洞见。代码已开源。
原文摘要 · Abstract (English)
Large Reasoning Models (LRMs) have achieved remarkable success, yet they often suffer from producing unnecessary and verbose reasoning chains. We identify a core aspect of this issue as "invalid thinking" -- models tend to repeatedly double-check their work after having derived the correct answer. To address this specific inefficiency, we move beyond the general principles of Efficacy and Efficiency to propose two new, fine-grained principles: Brevity, which advocates for eliminating redundancy, and Sufficiency, which ensures critical reasoning steps are preserved. Guided by these principles, we introduce LC-R1, a post-training method based on Group Relative Policy Optimization (GRPO). LC-R1 employs a novel combination of a Length Reward for overall conciseness and a Compress Reward that is specifically designed to remove the invalid portion of the thinking process. Extensive experiments on multiple reasoning benchmarks demonstrate that LC-R1 achieves a significant reduction in sequence length (~50%) with only a marginal (~2%) drop in accuracy, achieving a favorable trade-off point on the Pareto frontier that prioritizes high compression. Our analysis further validates the robustness of LC-R1 and provides valuable insights for developing more powerful yet computationally efficient LRMs. Our code is released at https://github.com/zxiangx/LC-R1.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。