发现大模型自带高效推理能力,用轻量方法引导其减少冗余输出。
Exploring and Exploiting the Inherent Efficiency within Large Reasoning Models for Self-Guided Efficiency Enhancement
- 通过分析模型内部表示,找到可引导的简洁推理方向。
- 在7个模型上缩短推理长度,准确率不降反升。
- 适合追求高效推理的落地应用开发者。
大型推理模型(LRMs)虽提升了复杂问题求解能力,但常因过度思考导致内容冗长,影响效率并增加推理成本。本文探究了该低效性的内在根源,发现正确推理路径长度差异显著,最短有效路径通常已足够,表明存在未开发的效率潜力。基于此,提出两种轻量级方法:一是无需训练的「效率导向」激活调控技术,通过模型表示空间中的单一方向调节推理行为;二是「自奖励效率强化学习」框架,通过奖励简洁正确的解法动态平衡任务准确率与表达简洁性。在多个数学推理基准上对七种不同骨干模型的实验表明,所提方法显著降低推理长度,同时保持或提升任务性能。结果证明,可通过自引导方式挖掘并利用现有模型的内在高效能力。
原文摘要 · Abstract (English)
Recent advancements in large reasoning models (LRMs) have significantly enhanced language models' capabilities in complex problem-solving by emulating human-like deliberative thinking. However, these models often exhibit overthinking (i.e., the generation of unnecessarily verbose and redundant content), which hinders efficiency and inflates inference cost. In this work, we explore the representational and behavioral origins of this inefficiency, revealing that LRMs inherently possess the capacity for more concise reasoning. Empirical analyses show that correct reasoning paths vary significantly in length, and the shortest correct responses often suffice, indicating untapped efficiency potential. Exploiting these findings, we propose two lightweight methods to enhance LRM efficiency. First, we introduce Efficiency Steering, a training-free activation steering technique that modulates reasoning behavior via a single direction in the model's representation space. Second, we develop Self-Rewarded Efficiency RL, a reinforcement learning framework that dynamically balances task accuracy and brevity by rewarding concise correct solutions. Extensive experiments on seven LRM backbones across multiple mathematical reasoning benchmarks demonstrate that our methods significantly reduce reasoning length while preserving or improving task performance. Our results highlight that reasoning efficiency can be improved by leveraging and guiding the intrinsic capabilities of existing models in a self-guided manner.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。