让大模型学会按需计算,省钱又提效。
Nice Fold or Hero Call: Learning Budget-Efficient Thinking for Adaptive Reasoning

- 基于预期回报动态分配算力,不靠难度预判
- 平均减少55%推理Token,性能还提升
- 适合需要高效推理的数学与逻辑任务
大型推理模型通过延长推理过程提升解题能力,但常在测试时错配计算资源。现有方法通过压缩推理轨迹或根据感知难度调控预算,却忽视了问题可解性。这导致模型对超出能力范围的问题过度投入算力,同时压缩本需深度推理的难题。本文将自适应推理建模为不确定下的计算投资,提出预算高效的思维(BET)框架:结合行为冷启动与GRPO,在考虑成本的奖励机制下训练。BET学习三种行为:(1)短解答,简洁回应简单问题;(2)优雅放弃,当继续推理预期收益趋近于零时提前停止;(3)英雄一击,为难而可解的问题保留足够算力。在七个基准和三个基础模型上,BET平均减少约55%推理令牌,整体性能提升,并实现从数学推理到科学问答与逻辑推理的零样本迁移,效率增益相当。
原文摘要 · Abstract (English)
Large reasoning models (LRMs) improve problem solving through extended reasoning, but often misallocate test-time compute. Existing efficiency methods reduce cost by compressing reasoning traces or conditioning budget on perceived difficulty, yet largely overlook solvability. As a result, they may spend large budgets on queries beyond the model's capability while compressing hard-but-solvable queries that require deeper reasoning. In this work, we formulate adaptive reasoning as a computational investment under uncertainty, where budget should follow the expected return of reasoning rather than perceived difficulty alone. To instantiate this principle, we propose Budget-Efficient Thinking (BET), a two-stage framework that combines behavioral cold-start with GRPO under an investment-cost-aware reward. By aligning solve-or-fold decisions with rollout-derived solvability, BET learns three behaviors: (1) short solve, answering easy queries concisely; (2) nice fold, abstaining early when continued reasoning has near-zero expected return; and (3) hero call, preserving sufficient compute for hard-but-solvable queries. Across seven benchmarks and three base models, BET reduces reasoning tokens by ~55% on average while achieving overall performance improvements, and transfers zero-shot from mathematical reasoning to scientific QA and logical reasoning with comparable efficiency gains.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。