通过区分内部与外部冗余,让大模型推理更简洁高效。
Reconsidering Overthinking: Penalizing Internal and External Redundancy in CoT Reasoning
- 将推理冗余分为思考停滞和答后赘述两类,分别惩罚。
- 在多个数据集上压缩40%以上长度,准确率基本不变。
- 适合追求推理效率与可解释性的研究者与工程师。
大型推理模型常出现过度思考,生成冗长的思维链(CoT)导致推理成本上升且过程难以理解。现有压缩方法多依赖全局长度奖励,混淆必要推理与冗余内容,可能损害推理准确性。本文从语义效率角度重新审视过思问题,将CoT冗余分解为两类:内部冗余指首次正确答案前的信息停滞;外部冗余指首次正确答案后的多余延续。基于此,提出双惩罚强化学习框架,分别优化推理进展与终止行为:使用滑动窗口语义相似度度量惩罚低进展段落,采用归一化外部冗余度量抑制答后续写。在GSM8K、MATH500和AIME24上的实验表明,该方法在1.5B模型上平均推理长度减少41.3%,7B模型减少40.1%,保持竞争力准确率,并在所有对比基线中取得最优的准确率-效率综合得分。所学压缩行为还可迁移至GPQA和LiveCodeBench等跨领域任务。分析显示两类冗余存在显著不对称性:外部冗余可大幅削减而性能损失小,内部冗余压缩则面临敏感的准确率-效率权衡。结果表明,有效的CoT压缩应聚焦语义效率而非单纯缩短序列长度,为构建更简洁、高效、可解释的大型推理模型提供系统性路径。
原文摘要 · Abstract (English)
Large reasoning models (LRMs) often exhibit overthinking, producing verbose Chain-of-Thought (CoT) traces that increase inference cost and obscure the underlying reasoning process. Existing CoT compression methods mainly rely on global length rewards, which conflate necessary intermediate reasoning with redundant text and may therefore compromise reasoning fidelity. This paper revisits overthinking from a semantic-efficiency perspective and decomposes CoT redundancy into two distinct forms: internal redundancy, defined as informational stagnation before the first correct answer, and external redundancy, defined as superfluous continuation after the first correct answer. Based on this decomposition, we propose a dual-penalty reinforcement learning framework that separately optimizes reasoning progress and termination behavior. Specifically, a sliding-window semantic similarity metric penalizes low-progress reasoning segments, while a normalized external-redundancy metric discourages post-answer continuation. Experiments on GSM8K, MATH500, and AIME24 across different model scales show that our method reduces average reasoning length by 41.3% on the 1.5B model and 40.1% on the 7B model, while preserving competitive accuracy and achieving the best overall accuracy-efficiency score among evaluated baselines. The learned compression behavior further transfers to out-of-domain reasoning tasks, including GPQA and LiveCodeBench. More importantly, our analysis reveals a clear asymmetry between the two redundancy types: external redundancy can be largely removed with little performance loss, whereas internal redundancy compression follows a sensitive accuracy-efficiency trade-off. These results suggest that effective CoT compression should optimize semantic efficiency rather than sequence length alone, offering a principled route toward more concise, efficient, and interpretable LRMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。