让大模型复用推理过程中的经验,提升计算效率
Do Not Waste Your Rollouts: Recycling Search Experience for Efficient Test-Time Scaling
- 将每次推理的中间结果存入共享经验库,供后续使用
- 在相同算力下,比现有方法在多个评测集上表现更优
- 无需训练,适合希望高效利用算力的推理场景
测试时扩展通过增加推理计算来拓宽解空间探索,但现有搜索策略通常将推理路径视为一次性消耗品,丢弃其中的中间洞察。这种浪费导致大量重复计算,模型反复推导已知结论、重访死胡同。为此,我们提出自引导、免训练的回收搜索经验(RSE)策略,将测试时搜索从孤立尝试转变为累积式经验驱动过程。通过主动将原始轨迹提炼为共享经验库,RSE实现正向经验复用(跳过冗余推导)与负向经验复用(剪枝已知死路)。理论上,我们形式化分析了RSE相较于独立采样的效率增益。实证上,在HMMT24、HMMT25、IMO-Bench和HLE上,RSE在同等计算预算下持续优于强基线,确立了测试时扩展的高效算力边界。
原文摘要 · Abstract (English)
Test-Time Scaling enhances the reasoning capabilities of Large Language Models by allocating additional inference compute to broaden the exploration of the solution space. However, existing search strategies typically treat rollouts as disposable samples, where valuable intermediate insights are effectively discarded after each trial. This wasted rollout-level experience leads to substantial computational redundancy, as models repeatedly re-derive discovered conclusions and revisit known dead ends across extensive attempts. To bridge this gap, we propose \textbf{Recycling Search Experience (RSE)}, a self-guided, training-free strategy that turns test-time search from a series of isolated trials into a cumulative, experience-guided process. By actively distilling raw trajectories into a shared experience bank, RSE enables positive recycling of intermediate conclusions to shortcut redundant derivations and negative recycling of failure patterns to prune encountered dead ends. Theoretically, we provide an analysis that formalizes the efficiency gains of RSE over independent sampling in solving complex reasoning tasks. Empirically, extensive experiments on HMMT24, HMMT25, IMO-Bench, and HLE show that RSE consistently outperforms strong baselines under comparable computational budgets, establishing a strong compute-efficiency frontier for test-time scaling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。