动态分配推理计算，让模型自己选最优解题路径。
What If We Allocate Test-Time Compute Adaptively?
- 用验证器引导的迭代生成，自动调整每步推理策略。
- 在MATH-500等难题上性能显著提升，部分任务增益数倍。
- 适合追求高精度推理、重视计算效率的研究者。
测试时计算扩展通常均匀分配计算资源，采用固定采样策略，并仅在重排序阶段进行验证。本文提出一种验证器引导的自适应框架，将推理视为迭代轨迹生成与选择过程。对于每个问题，智能体运行多轮推理迭代：每轮可生成高层计划，联合选择推理工具、计算策略及探索参数，并生成候选推理轨迹。过程奖励模型（PRM）作为统一控制信号：在单轮内，步骤级PRM得分用于指导生成过程中的剪枝与扩展；跨轮次间，轨迹总分用于选取最终答案。在多个数据集上，该动态、PRM引导的方法持续优于直接的测试时扩展，在MATH-500上取得显著提升，并在更难的AIME24和AMO-Bench上实现数倍改进。通过理论浮点运算量（FLOPs）和计算强度指标（惩罚无效生成与工具开销），证明验证引导的分配能聚焦计算于高价值推理路径。
原文摘要 · Abstract (English)
Test-time compute scaling allocates inference computation uniformly, uses fixed sampling strategies, and applies verification only for reranking. In contrast, we propose a verifier-guided adaptive framework treating reasoning as iterative trajectory generation and selection. For each problem, the agent runs multiple inference iterations. In each iteration, it optionally produces a high-level plan, selects a set of reasoning tools and a compute strategy together with an exploration parameter, and then generates a candidate reasoning trajectory. A process reward model (PRM) serves as a unified control signal: within each iteration, step-level PRM scores are aggregated to guide pruning and expansion during generation, and across iterations, aggregated trajectory rewards are used to select the final response. Across datasets, our dynamic, PRM-guided approach consistently outperforms direct test-time scaling, yielding large gains on MATH-500 and several-fold improvements on harder benchmarks such as AIME24 and AMO-Bench. We characterize efficiency using theoretical FLOPs and a compute intensity metric penalizing wasted generation and tool overhead, demonstrating that verification-guided allocation concentrates computation on high-utility reasoning paths.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。