用小模型生成的思考路径,让大模型少想多做,省时省力。
Thought Manipulation: External Thought Can Be Efficient for Large Reasoning Models
- 在思考标记间插入小模型生成的思维链,引导大模型减少冗余推理
- 应用后输出令牌数减少约30%,性能基本不变,额外开销极小
- 适合希望降低大模型推理成本的研究者与开发者
大型推理模型(LRMs)通过增加测试时计算量,在多项任务中展现了强大推理能力。然而,这些模型常出现‘过度思考’问题,即生成大量冗余推理步骤却收益有限。本文实证发现,在之间插入由小型模型生成的外部思维链(CoT),可有效引导大模型减少思考次数。基于此,我们提出一种简单高效的流水线方法 extMethod,使大模型跳过不必要的中间步骤,显著降低计算成本。在QwQ-32B模型上对LiveBench/Code数据集的实验表明,该方法在保持原性能的同时,输出令牌数减少约30%,且外部思维链生成器引入的开销极小。此外,我们识别出两种次优行为:盲目遵循错误的外部思考、无意义的重新思考,并证明通过难度感知的回退策略可进一步提升效果。整体而言, extMethod为大型推理模型推理提供了实用、通用且高效优化方案,助力其在真实场景中更广泛部署。
原文摘要 · Abstract (English)
Recent advancements in large reasoning models (LRMs) have demonstrated the effectiveness of scaling test-time computation to enhance reasoning capabilities on various tasks. However, LRMs often suffer from an ``overthinking'' problem, where the model generates excessively redundant reasoning steps with limited performance gains. In this work, we empirically reveal an important characteristic of LRM behaviors that placing external CoTs generated by smaller models between the thinking token (\texttt{<think>} and \texttt{</think>}) can effectively manipulate the model to generate fewer thoughts. Building on this finding, we propose a simple yet efficient pipeline, \Method, to enable LRMs to bypass unnecessary intermediate steps, thereby significantly reducing computational costs. We conduct extensive experiments to evaluate the utility and efficiency of \Method. For instance, when applied to QwQ-32B on the LiveBench/Code dataset, \Method keeps the original performance while reducing output token counts by approximately 30\%, with minimal overhead introduced by the CoT generator. Furthermore, we identify two suboptimal modes, blindly following flawed external thoughts and unnecessary rethinking, and show that simple mitigations, such as difficulty-aware fallbacks, can further improve performance. Overall, \Method offers a practical, general, and efficient way to optimize LRM inference, making powerful reasoning models more accessible and scalable for real-world applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。