arXiv:2601.02972cs.CLcs.AI2026-01ACL被引 3

让大模型推理更短更准,避免无意义的过度思考。

Correct, Concise and Complete: Multi-stage Training For Adaptive Reasoning

  • 分阶段训练结合监督微调与自适应长度惩罚,控制推理过程。
  • 8B模型响应缩短28%,32B模型缩短40%,准确率仅小幅下降。
  • 适合追求高效推理、降低算力消耗的研究者和开发者。

大语言模型的推理能力通过增加测试时计算(如思维链)显著提升,但思维链常过长,导致计算成本上升却未带来准确率提升,甚至可能降低性能,称为“过度思考”。本文提出一种多阶段高效推理方法,结合监督微调(通过拒绝采样或推理路径重构)与强化学习,采用自适应长度惩罚机制。设计轻量级奖励函数,在首次正确答案后惩罚多余生成,仅在自我验证有益时鼓励其进行。在七个不同推理任务上进行综合评估,分析准确率与响应长度的权衡。该方法使8B模型平均响应长度减少28%,32B模型减少40%,准确率仅下降1.6和2.5点。尽管概念简单,其在修正过度思考后的准确率曲线下面积(AUC_OAA)达到76.6,优于基线模型5点,次优方法2.5点。

原文摘要 · Abstract (English)

The reasoning capabilities of large language models (LLMs) have improved substantially through increased test-time computation, typically in the form of intermediate tokens known as chain-of-thought (CoT). However, CoT often becomes unnecessarily long, increasing computation cost without actual accuracy gains or sometimes even degrading performance, a phenomenon known as ``overthinking''. We propose a multi-stage efficient reasoning method that combines supervised fine-tuning -- via rejection sampling or reasoning trace reformatting -- with reinforcement learning using an adaptive length penalty. We introduce a lightweight reward function that penalizes tokens generated after the first correct answer but encouraging self-verification only when beneficial. We conduct a holistic evaluation across seven diverse reasoning tasks, analyzing the accuracy-response length trade-off. Our approach reduces response length by an average of 28\% for 8B models and 40\% for 32B models, while incurring only minor performance drops of 1.6 and 2.5 points, respectively. Despite its conceptual simplicity, it achieves a superior trade-off compared to more complex state-of-the-art efficient reasoning methods, scoring 76.6, in terms of the area under the Overthinking-Adjusted Accuracy curve ($\text{AUC}_{\text{OAA}}$) -- 5 points above the base model and 2.5 points above the second-best approach.

推理优化大模型思维链效率提升

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。