长推理不等于好结果,短而准的思考更有效。
When More is Less: Understanding Chain-of-Thought Length in LLMs
- 发现推理步骤越多,准确率先升后降,呈倒U型曲线。
- 越难的任务需要更长推理,但越强模型反而偏好短推理。
- 提出可指导训练与推理的长度自适应策略,避免过度思考。
大型语言模型常使用思维链(CoT)分解复杂问题。尽管普遍认为更长的思维链更优,本文挑战这一观点,指出长并不总是好。结合真实观察、受控实验与理论分析,我们发现任务准确率随思维链长度变化呈倒U型曲线:初期提升,随后下降。受控实验显示,最优思维链长度随任务难度增加而增长,但随模型能力增强而缩短,揭示出模型内在的简洁性偏好。此偏好在强化学习训练中也可见,模型准确率提高时趋向更短的思维链。我们构建一个简单理论模型,形式化证明了这些现象及最优长度的缩放规律,并解释了强化学习中简洁性偏好的出现。基于该框架,我们验证了使用最优长度思维链训练与推理阶段长度感知过滤带来的显著收益。研究为‘过度思考’现象提供了原理性理解,并给出可落地的思维链校准指南,使模型能根据任务复杂度与自身能力动态调整推理长度,实现最优推理性能。
原文摘要 · Abstract (English)
Large Language Models (LLMs) employ Chain-of-Thought (CoT) reasoning to deconstruct complex problems. While longer CoTs are often presumed superior, this paper challenges that notion, arguing that longer is not always better. Drawing on combined evidence from real-world observations, controlled experiments, and theoretical analysis, we demonstrate that task accuracy typically follows an inverted U-shaped curve with CoT length, where performance initially improves but eventually decreases as the number of CoT steps increases. With controlled experiments, we further uncover the scaling behaviors of the optimal CoT length: it increases with task difficulty but decreases with model capability, exposing an inherent simplicity bias where more capable models favor shorter, more efficient CoT reasoning. This bias is also evident in Reinforcement Learning (RL) training, where models gravitate towards shorter CoTs as their accuracy improves. To have a deep understanding of these dynamics, we establish a simple theoretical model that formally proves these phenomena, including the optimal length's scaling laws and the emergence of simplicity bias during RL. Guided by this framework, we demonstrate significant practical benefits from training with optimally-lengthed CoTs and employing length-aware filtering at inference. These findings offer both a principled understanding of the "overthinking" phenomenon and multiple practical guidelines for CoT calibration, enabling LLMs to achieve optimal reasoning performance with adaptive CoTs tailored to task complexity and model capability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。