arXiv:2509.25420cs.AIcs.CL2025-09被引 2

让大模型分两步走:先规划再执行,更高效地推理

Adaptive Test-Time Reasoning via Reward-Guided Dual-Phase Search

  • 把推理拆成规划和执行两阶段,分别搜索优化
  • 动态分配计算资源,难的地方多花力气,快完成的提前停
  • 在数学和代码任务上既更准又省算力

大型语言模型在推理任务中取得显著进展。主流方法是基于树的搜索结合验证器,通过扩展候选推理路径并利用奖励模型指导剪枝与选择。尽管有效提升准确率,但效率不足:这些方法对推理过程进行简单分解,忽略了数学推理或代码生成等任务中计划-执行的特性,导致推理探索效率低下。为此,我们提出一种双阶段测试时扩展框架,显式将推理分为规划与执行两个阶段,并分别对两阶段进行搜索。具体而言,我们将推理轨迹分解,为每个阶段开发奖励模型,使搜索能独立探索与剪枝计划和执行过程。此外,引入动态预算分配机制,根据奖励反馈自适应重分配采样资源,对信心高的步骤实现早期停止,并将计算力重新分配至更困难的推理环节。在数学推理与代码生成基准上的实验表明,该方法在持续提升准确率的同时,显著减少冗余计算。

原文摘要 · Abstract (English)

Large Language Models (LLMs) have achieved significant advances in reasoning tasks. A key approach is tree-based search with verifiers, which expand candidate reasoning paths and use reward models to guide pruning and selection. Although effective in improving accuracy, these methods are not optimal in terms of efficiency: they perform simple decomposition on the reasoning process, but ignore the planning-execution nature of tasks such as math reasoning or code generation. This results in inefficient exploration of reasoning process. To address this, we propose a dual-phase test-time scaling framework that explicitly separates reasoning into planning and execution, and performs search over the two phases individually. Specifically, we decompose reasoning trajectories and develop reward models for each phase, enabling the search to explore and prune plans and executions separately. We further introduce a dynamic budget allocation mechanism that adaptively redistributes sampling effort based on reward feedback, allowing early stopping on confident steps and reallocation of computation to more challenging parts of the reasoning process. Experiments on both mathematical reasoning and code generation benchmarks demonstrate that our approach consistently improves accuracy while reducing redundant computation.

大模型推理双阶段搜索测试时扩展动态调度

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。