改进树搜索算法，让大模型推理更准且越算越好
Revisiting Tree Search for LLMs: Gumbel and Sequential Halving for Budget-Scalable Reasoning
- 用Gumbel采样和分阶段筛选替代原有策略，提升搜索效率
- 在GSM8K上达58.4%准确率，Game24上达85.3%，预算越大越准
- 无需改模型或训练，适合想提升推理能力的开发者
神经树搜索是游戏博弈和基于模型强化学习等复杂领域中的强大决策算法。近期研究将类似AlphaZero的树搜索应用于大语言模型（LLMs）的推理过程，以增强其推理能力，但我们发现该方法存在缩放失效问题：随着搜索预算增加，GSM8K和Game24上的准确率反而下降。本文提出ReSCALE，一种对Gumbel AlphaZero MCTS的改进方法，用Gumbel采样和分阶段筛选替代狄利克雷噪声与PUCT选择，无需修改模型或训练即可恢复单调可扩展性。ReSCALE在搜索预算较高时分别达到GSM8K上58.4%和Game24上85.3%的准确率，优于基线。消融实验表明，分阶段筛选是性能提升的主要原因。
原文摘要 · Abstract (English)
Neural tree search is a powerful decision-making algorithm widely used in complex domains such as game playing and model-based reinforcement learning. Recent work has applied AlphaZero-style tree search to enhance the reasoning capabilities of Large Language Models (LLMs) during inference, but we find that this approach suffers from a scaling failure: on GSM8K and Game24, accuracy drops as the search budget increases. In this paper, we present ReSCALE, an adaptation of Gumbel AlphaZero MCTS that replaces Dirichlet noise and PUCT selection with Gumbel sampling and Sequential Halving, restoring monotonic scaling without changes to the model or its training. ReSCALE reaches 58.4\% on GSM8K and 85.3\% on Game24 at budgets where the baseline degrades. Ablations confirm that Sequential Halving is the primary driver of the improvement.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。