arXiv:2505.14604cs.CLcs.AI2025-05NeurIPS被引 27

让大模型自己学会适时停止推理,减少冗余计算。

Let LRMs Break Free from Overthinking via Self-Braking Tuning

  • 设计自检机制识别推理中的冗余步骤,生成自我调节训练信号。
  • 在数学基准上降低60%的令牌消耗,精度与原模型相当。
  • 适合追求高效推理的AI系统开发者和部署优化者。

大型推理模型(如OpenAI o1和DeepSeek-R1)通过生成更长的思维链显著提升了性能,但伴随而来的是大量冗余推理,导致计算开销高企并加剧了过度思考问题。现有方法多依赖外部干预,本文提出自制动调优(SBT)框架,让模型自主调控推理过程。基于标准答案构建过度思考识别指标,系统检测推理轨迹中的无效步骤,并生成训练信号以学习自我调节。在此基础上,开发出具有自适应推理长度的数据构建策略,引入创新的制动提示机制,使模型自然学会在合适时机终止推理。在数学基准(AIME、AMC、MATH500、GSM8K)上的实验表明,该方法可将令牌消耗降低高达60%,同时保持与无约束模型相当的准确率。

原文摘要 · Abstract (English)

Large reasoning models (LRMs), such as OpenAI o1 and DeepSeek-R1, have significantly enhanced their reasoning capabilities by generating longer chains of thought, demonstrating outstanding performance across a variety of tasks. However, this performance gain comes at the cost of a substantial increase in redundant reasoning during the generation process, leading to high computational overhead and exacerbating the issue of overthinking. Although numerous existing approaches aim to address the problem of overthinking, they often rely on external interventions. In this paper, we propose a novel framework, Self-Braking Tuning (SBT), which tackles overthinking from the perspective of allowing the model to regulate its own reasoning process, thus eliminating the reliance on external control mechanisms. We construct a set of overthinking identification metrics based on standard answers and design a systematic method to detect redundant reasoning. This method accurately identifies unnecessary steps within the reasoning trajectory and generates training signals for learning self-regulation behaviors. Building on this foundation, we develop a complete strategy for constructing data with adaptive reasoning lengths and introduce an innovative braking prompt mechanism that enables the model to naturally learn when to terminate reasoning at an appropriate point. Experiments across mathematical benchmarks (AIME, AMC, MATH500, GSM8K) demonstrate that our method reduces token consumption by up to 60% while maintaining comparable accuracy to unconstrained models.

推理优化大模型自调节效率提升

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。