分析大模型如何省思考，发现三种推理模式及效率与准确率的权衡。
When Can Large Reasoning Models Save Thinking? Mechanistic Analysis of Behavioral Divergence in Reasoning
- 通过注意力机制识别出无思考、显式思考和隐式思考三种模式。
- 无思考模式缩短输出但降低准确率，另两种在保持准确率前提下减少长度。
- 揭示强化学习优化模型内在矛盾，适合关注推理效率的研究者阅读。
大推理模型（LRMs）在复杂任务上表现显著提升，但过度思考导致效率低下。本研究分析了强化学习（RL）训练的LRMs在被要求节省思考时的内部机制，发现三种不同的思考模式：无思考（NT）、显式思考（ET）和隐式思考（IT）。通过分析思考终止置信度、思考到生成的注意力转移以及对输入部分的关注度，我们揭示了影响推理行为的关键因素。进一步发现，NT模式虽能缩短输出长度，但以牺牲准确率为代价；而ET和IT模式在保持准确率的同时有效缩减响应长度。研究结果暴露了RL优化的LRMs中存在根本性不一致，亟需自适应改进以实现可靠高效推理。
原文摘要 · Abstract (English)
Large reasoning models (LRMs) have significantly advanced performance on complex tasks, yet their tendency to overthink introduces inefficiencies. This study investigates the internal mechanisms of reinforcement learning (RL)-trained LRMs when prompted to save thinking, revealing three distinct thinking modes: no thinking (NT), explicit thinking (ET), and implicit thinking (IT). Through comprehensive analysis of confidence in thinking termination, attention from thinking to generation, and attentional focus on input sections, we uncover key factors influencing the reasoning behaviors. We further find that NT reduces output length at the cost of accuracy, while ET and IT maintain accuracy with reduced response length. Our findings expose fundamental inconsistencies in RL-optimized LRMs, necessitating adaptive improvements for reliable efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。