让大模型学会判断何时该深度思考，提升推理效率
Thinkless: LLM Learns When to Think
- 用两个控制标记自动选择简洁或深度推理路径
- 在多个数学题集上减少50%~90%的长链思考使用
- 适合追求高效推理的大模型应用开发者
具备链式思维能力的推理语言模型在需要复杂逻辑推理的任务中表现卓越。然而，对所有问题都采用复杂推理会带来显著的计算开销，尤其当许多问题其实有简单解法时。这引发一个核心问题：大模型能否学会何时该思考？为此，我们提出Thinkless，一种可学习的框架，使大模型能根据任务复杂度和自身能力自适应地在简短回答与详细推理之间切换。Thinkless采用强化学习训练，使用<short>表示简洁回应，<think>表示深度推理。其核心是解耦分组相对策略优化（DeGRPO）算法，将混合推理的学习目标分解为两部分：（1）控制标记损失，用于决策推理模式；（2）响应损失，用于提升答案准确性。这种解耦设计实现了对各目标贡献的精细控制，稳定了训练过程，并有效避免了传统GRPO中的崩溃问题。实验证明，在Minerva Algebra、MATH-500和GSM8K等多个基准测试中，Thinkless可将长链思考使用量降低50%至90%，显著提升推理模型效率。
原文摘要 · Abstract (English)
Reasoning Language Models, capable of extended chain-of-thought reasoning, have demonstrated remarkable performance on tasks requiring complex logical inference. However, applying elaborate reasoning for all queries often results in substantial computational inefficiencies, particularly when many problems admit straightforward solutions. This motivates an open question: Can LLMs learn when to think? To answer this, we propose Thinkless, a learnable framework that empowers an LLM to adaptively select between short-form and long-form reasoning, based on both task complexity and the model's ability. Thinkless is trained under a reinforcement learning paradigm and employs two control tokens, <short> for concise responses and <think> for detailed reasoning. At the core of our method is a Decoupled Group Relative Policy Optimization (DeGRPO) algorithm, which decomposes the learning objective of hybrid reasoning into two components: (1) a control token loss that governs the selection of the reasoning mode, and (2) a response loss that improves the accuracy of the generated answers. This decoupled formulation enables fine-grained control over the contributions of each objective, stabilizing training and effectively preventing collapse observed in vanilla GRPO. Empirically, on several benchmarks such as Minerva Algebra, MATH-500, and GSM8K, Thinkless is able to reduce the usage of long-chain thinking by 50% - 90%, significantly improving the efficiency of Reasoning Language Models. The code is available at https://github.com/VainF/Thinkless
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。