发现大模型推理中冗余自我确认步骤,可有效压缩输出长度而不降精度。
Efficient Reasoning Through Suppression of Self-Affirmation Reflections in Large Reasoning Models
- 识别出推理中的冗余自我确认反射,通过首词概率偏差定位
- 无训练情况下输出长度减少18.7%,有训练时达50.2%压缩率
- 方法简单易用,可直接集成到vLLM等推理框架
尽管大型推理模型性能显著提升,但输出长度快速增长仍制约其效率。现有优化方法存在‘过度思考’倾向,却缺乏细粒度分析。本文聚焦于‘自我确认反射’——在已正确推理后重复确认内容的冗余反思步骤。对原始与优化模型的观察显示,此类反射普遍存在,甚至导致优化模型输出更长。深入分析发现,自我确认反射的句子首词具有独特的概率偏差。基于此,我们提出无需训练的方法定位并抑制此类反射,在R1-Distill-Models、QwQ-32B和Qwen3-32B上实现输出长度压缩,且不降低准确率。此外,还改进了现有训练方法,显式抑制此类反射。实验表明,无训练设置下长度压缩18.7%,训练设置下达50.2%(针对R1-Distill-Qwen-1.5B)。该方法简洁实用,可直接应用于vLLM等推理框架。
原文摘要 · Abstract (English)
While recent advances in large reasoning models have demonstrated remarkable performance, efficient reasoning remains critical due to the rapid growth of output length. Existing optimization approaches highlights a tendency toward "overthinking", yet lack fine-grained analysis. In this work, we focus on Self-Affirmation Reflections: redundant reflective steps that affirm prior content and often occurs after the already correct reasoning steps. Observations of both original and optimized reasoning models reveal pervasive self-affirmation reflections. Notably, these reflections sometimes lead to longer outputs in optimized models than their original counterparts. Through detailed analysis, we uncover an intriguing pattern: compared to other reflections, the leading words (i.e., the first word of sentences) in self-affirmation reflections exhibit a distinct probability bias. Motivated by this insight, we can locate self-affirmation reflections and conduct a train-free experiment demonstrating that suppressing self-affirmation reflections reduces output length without degrading accuracy across multiple models (R1-Distill-Models, QwQ-32B, and Qwen3-32B). Furthermore, we also improve current train-based method by explicitly suppressing such reflections. In our experiments, we achieve length compression of 18.7\% in train-free settings and 50.2\% in train-based settings for R1-Distill-Qwen-1.5B. Moreover, our improvements are simple yet practical and can be directly applied to existing inference frameworks, such as vLLM. We believe that our findings will provide community insights for achieving more precise length compression and step-level efficient reasoning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。