arXiv:2505.16122cs.LG2025-05中稿 · ICLR被引 34

通过拆解问题动态分配计算资源,让大模型推理更高效准确。

Plan and Budget: Effective and Efficient Test-Time Scaling on Reasoning Large Language Models

  • 将复杂问题分解为子问题,按难度动态分配计算预算。
  • 在多个任务上实现70%准确率提升、39%令牌减少、E3指标提高193.8%。
  • 无需重训练即可让小模型效率逼近大模型,适合部署优化场景。

大型语言模型在复杂推理任务中表现优异,但推理过程计算效率低下。我们发现许多主流模型存在过度思考现象:即使面对简单问题也生成冗长且离题的推理链。已有工作尝试通过固定令牌预算缓解此问题,但可能导致在难题上思考不足。通过实证分析,我们识别出该低效主要源于缺乏清晰的问题求解策略。为此,我们构建了理论模型BAM(Budget Allocation Model),将推理建模为一系列不确定性各异的子问题,并提出E3指标以衡量正确性与计算效率之间的权衡。基于BAM理论结果,我们提出Plan-and-Budget——一种模型无关的测试时框架,将复杂查询分解为子问题,并利用自适应调度根据估计复杂度分配令牌预算。该方法在多种任务和模型上均提升了推理效率,最高实现70%准确率增益、39%令牌减少、193.8% E3改进。尤为关键的是,它使小型模型(DS-Qwen-32B)的效率达到大型模型(DS-LLaMA-70B)水平,证明其可在不重新训练的前提下缩小性能差距。代码已开源:https://github.com/junhongmit/P-and-B。

原文摘要 · Abstract (English)

Large Language Models (LLMs) have achieved remarkable success in complex reasoning tasks, but their inference remains computationally inefficient. We observe a common failure mode in many prevalent LLMs, overthinking, where models generate verbose and tangential reasoning traces even for simple queries. Recent work has tried to mitigate this by enforcing fixed token budgets, however, this can lead to underthinking, especially on harder problems. Through empirical analysis, we identify that this inefficiency often stems from unclear problem-solving strategies. To formalize this, we develop a theoretical model, BAM (Budget Allocation Model), which models reasoning as a sequence of sub-questions with varying uncertainty, and introduce the E3 metric to capture the trade-off between correctness and computation efficiency. Building on theoretical results from BAM, we propose Plan-and-Budget, a model-agnostic, test-time framework that decomposes complex queries into sub-questions and allocates token budgets based on estimated complexity using adaptive scheduling. Plan-and-Budget improves reasoning efficiency across a range of tasks and models, achieving up to 70% accuracy gains, 39% token reduction, and 193.8% improvement in E3. Notably, it improves the efficiency of a smaller model (DS-Qwen-32B) to match the efficiency of a larger model (DS-LLaMA-70B), demonstrating Plan-and-Budget's ability to close performance gaps without retraining. Our code is available at https://github.com/junhongmit/P-and-B.

推理优化测试时缩放大模型效率

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。