智能调节推理长度，让大模型既省力又更准。
SmartThinker: Progressive Chain-of-Thought Length Calibration for Efficient Large Language Model Reasoning
- 动态调整推理路径长度，训练时自动找最优长度点。
- 实现平均52.5%的输出压缩，挑战题准确率提升16.6%。
- 适合追求高效精准推理的AI研究者与工程师使用。
大型推理模型（LRM）如OpenAI o1和DeepSeek-R1通过长链式思维（CoT）路径在复杂任务中取得高精度，但过程常冗余且过度思考。现有方法采用组相对策略优化（GRPO）减少输出长度，但静态长度奖励设计无法随问题难度和响应长度分布动态调整，导致过度压缩并损害准确率。为此，我们提出SmartThinker，一种基于GRPO的高效推理方法，具备双重创新：首先，在训练中动态估计达到峰值准确率的最优长度，并引导过长响应向其收敛，降低长度同时保持准确率；其次，动态调节长度奖励系数，避免对正确推理路径的误惩罚。大量实验表明，SmartThinker实现平均52.5%的长度压缩，且在挑战性基准AIME25上准确率最高提升16.6%。源码见https://github.com/SJTU-RTEAS/SmartThinker。
原文摘要 · Abstract (English)
Large reasoning models (LRMs) like OpenAI o1 and DeepSeek-R1 achieve high accuracy on complex tasks by adopting long chain-of-thought (CoT) reasoning paths. However, the inherent verbosity of these processes frequently results in redundancy and overthinking. To address this issue, existing works leverage Group Relative Policy Optimization (GRPO) to reduce LRM output length, but their static length reward design cannot dynamically adapt according to the relative problem difficulty and response length distribution, causing over-compression and compromised accuracy. Therefore, we propose SmartThinker, a novel GRPO-based efficient reasoning method with progressive CoT length calibration. SmartThinker makes a two-fold contribution: First, it dynamically estimates the optimal length with peak accuracy during training and guides overlong responses toward it to reduce response length while sustaining accuracy. Second, it dynamically modulates the length reward coefficient to avoid the unwarranted penalization of correct reasoning paths. Extensive experiment results show that SmartThinker achieves up to 52.5% average length compression with improved accuracy, and achieves up to 16.6% accuracy improvement on challenging benchmarks like AIME25. The source code can be found at https://github.com/SJTU-RTEAS/SmartThinker.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。