通过自信度抑制冗余反思,让大模型推理更省 token 又不丢准确率。
Efficient Reasoning for Large Reasoning Language Models via Certainty-Guided Reflection Suppression
- 根据模型自信度动态关闭反思触发词,避免无效思考循环。
- 在多个数学与推理数据集上平均减少 18.5%~41.9% 的生成 token。
- 无需重训练或改结构,适配主流大模型且跨规模稳定有效。
近期的大规模推理语言模型(LRLMs)采用长思维链与复杂反思行为,通常以特定触发词(如"Wait"和"Alternatively")标识,以提升性能。然而,这些反思行为常引发过度思考问题:产生冗余推理步骤,增加令牌使用量、推高推理成本、降低实用性。本文提出一种名为自信度引导的反思抑制(CGRS)的新方法,在保持推理准确性的前提下缓解过度思考。CGRS通过在模型对当前回答高度自信时动态抑制反思触发词的生成,防止冗余反思循环而不损害输出质量。该方法具备模型无关性,无需重训练或架构修改,可无缝集成至现有自回归生成流程。在四个推理基准测试(AIME24、AMC23、MATH500、GPQA-D)上的大量实验表明,CGRS平均减少 18.5% 至 41.9% 的令牌用量,同时保持准确率。其在长度压缩与性能间取得最优平衡,优于现有最先进基线。该效果在不同模型架构(如 DeepSeek-R1-Distill 系列、QwQ-32B、Qwen3 家族)及规模(4B 到 32B 参数)下均一致成立,凸显其在高效推理中的实用价值。
原文摘要 · Abstract (English)
Recent Large Reasoning Language Models (LRLMs) employ long chain-of-thought reasoning with complex reflection behaviors, typically signaled by specific trigger words (e.g., "Wait" and "Alternatively") to enhance performance. However, these reflection behaviors can lead to the overthinking problem where the generation of redundant reasoning steps that unnecessarily increase token usage, raise inference costs, and reduce practical utility. In this paper, we propose Certainty-Guided Reflection Suppression (CGRS), a novel method that mitigates overthinking in LRLMs while maintaining reasoning accuracy. CGRS operates by dynamically suppressing the model's generation of reflection triggers when it exhibits high confidence in its current response, thereby preventing redundant reflection cycles without compromising output quality. Our approach is model-agnostic, requires no retraining or architectural modifications, and can be integrated seamlessly with existing autoregressive generation pipelines. Extensive experiments across four reasoning benchmarks (i.e., AIME24, AMC23, MATH500, and GPQA-D) demonstrate CGRS's effectiveness: it reduces token usage by an average of 18.5% to 41.9% while preserving accuracy. It also achieves the optimal balance between length reduction and performance compared to state-of-the-art baselines. These results hold consistently across model architectures (e.g., DeepSeek-R1-Distill series, QwQ-32B, and Qwen3 family) and scales (4B to 32B parameters), highlighting CGRS's practical value for efficient reasoning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。