优化大模型推理时的计算资源分配,提升数学推理准确率
Every Rollout Counts: Optimal Resource Allocation for Efficient Test-Time Scaling
- 按推理方向分配计算资源,而非按候选解数量
- 在MATH500等基准上实现当前最优准确率
- 适合关注推理效率与性能提升的研究者
测试时扩展(TTS)通过增加推理时计算量,探索多条推理路径来提升大语言模型性能。然而,如何在固定回滚预算下高效分配计算资源仍未被充分研究,常导致计算资源浪费。本文将测试时搜索建模为资源分配问题,推导出在固定预算下最大化正确解获得概率的最优策略。分析揭示现有方法存在根本缺陷:基于解级别的分配偏好候选数多的方向,造成理论上次优且低效的资源使用。为此,提出方向导向资源分配(DORA),通过解耦方向质量与候选数量,在方向层面进行资源分配,实现理论最优。在MATH500、AIME2024和AIME2025等挑战性数学推理基准上广泛实验表明,DORA以相近计算成本持续优于强基线,达到当前最优准确率。研究希望推动对大模型测试时扩展最优性的深入理解。
原文摘要 · Abstract (English)
Test-Time Scaling (TTS) improves the performance of Large Language Models (LLMs) by using additional inference-time computation to explore multiple reasoning paths through search. Yet how to allocate a fixed rollout budget most effectively during search remains underexplored, often resulting in inefficient use of compute at test time. To bridge this gap, we formulate test-time search as a resource allocation problem and derive the optimal allocation strategy that maximizes the probability of obtaining a correct solution under a fixed rollout budget. Within this formulation, we reveal a core limitation of existing search methods: solution-level allocation tends to favor reasoning directions with more candidates, leading to theoretically suboptimal and inefficient use of compute. To address this, we propose Direction-Oriented Resource Allocation (DORA), a provably optimal method that mitigates this bias by decoupling direction quality from candidate count and allocating resources at the direction level. To demonstrate DORA's effectiveness, we conduct extensive experiments on challenging mathematical reasoning benchmarks including MATH500, AIME2024, and AIME2025. The empirical results show that DORA consistently outperforms strong baselines with comparable computational cost, achieving state-of-the-art accuracy. We hope our findings contribute to a broader understanding of optimal TTS for LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。