通过回溯历史生成路径,用更少的计算量提升大模型推理准确率。
Beyond the Frontier: Stochastic Backtracking for Efficient Test-Time Scaling

- 保留历史生成前缀池,允许回溯旧路径而非只推进当前前沿。
- 在数学推理任务上,相同准确率下仅需1/3~1/2的生成词数。
- 适合追求高精度低耗时推理的开发者和研究者。
测试时扩展通过额外计算探索多个解题路径来提升语言模型的推理能力。核心挑战是在保证准确率的同时最小化生成的总词数。现有基于评分函数(PRM)的方法仅保留当前活跃前缀,利用有噪声的评分不可逆地剪枝或重采样其余路径,可能导致过早决策、多样性丧失以及遗漏仍可能通向正确结果的前缀。本文提出一种持久化历史前缀池的随机回溯机制,使测试时计算可重新访问之前生成的状态,而不仅局限于扩展当前前沿。为提高效率,设计两种互补机制:子池选择通过在随机子池中进行Top-N筛选,增强贪心搜索,让历史前缀有机会绕过被高估的前沿候选;幂次回溯序列蒙特卡洛将SMC式重采样引入持久池,使用加权后的幂次评分与混合校正权重。在多个数学推理基准和不同模型规模下,该方法始终以更低的词数实现更高准确率,相同准确率所需词数仅为强基线的1/3至1/2,证明了持久池随机回溯是提升测试时扩展中准确率-词数权衡的简单有效方案。
原文摘要 · Abstract (English)
Test-time scaling improves language model reasoning by spending additional compute to explore multiple solution trajectories. The key challenge is to maximize accuracy while minimizing the total number of generated tokens during reasoning. Recent PRM-guided methods score intermediate prefixes to steer this search, but most are frontier-only: they keep only the current active prefixes and irreversibly prune or resample away the rest using noisy PRM scores. This can cause premature commitment, diversity collapse, and the loss of prefixes that still admit correct continuations. We introduce stochastic backtracking over a persistent pool of historical prefixes, allowing test-time compute to revisit previously generated states instead of only expanding the current frontier. To make this efficient, we propose two complementary mechanisms. Subpool Selection strengthens greedy PRM-guided search by applying Top-N selection within random subpools, giving historical prefixes a chance to bypass over-scored frontier candidates. Power Backtrack Sequential Monte Carlo extends SMC-style resampling to the persistent pool using powered PRM scores and mixture-corrected weights. Across mathematical reasoning benchmarks and model scales, our methods consistently achieve higher accuracy per token count, and the same level of accuracy using only a fraction of the token count in comparison to strong PRM-guided baselines, demonstrating that persistent-pool stochastic backtracking provides a simple and effective way to improve the accuracy-token trade-off in test-time scaling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。