通过粒子采样提升大模型推理效率,让计算资源更智能地分配给有潜力的解题路径。
The Model Knows, the Decoder Finds: Future Value Guided Particle Power Sampling

- 采用分块粒子算法,用重加权和未来价值引导选择来优化候选路径
- 在不训练的情况下,实现比传统方法更高的推理准确率,且内存可预测
- 适合追求高精度推理但受限于算力的场景,如部署阶段优化
在无需训练的推理中,基础大语言模型已对正确多步解赋予非平凡概率,瓶颈在于推理时高效定位这些解。幂采样通过目标分布 p_theta(x)^alpha(α>1)实现对解空间的合理偏置,但实际近似需考虑依赖未来的修正因子以判断哪些前缀仍具前景。本文提出辅助粒子幂采样(APPS),一种基于块级粒子的近似方法,用有限数量的部分解逼近序列级幂目标。APPS 通过提案校正的幂重加权并行传播假设,并在重采样边界使用未来价值引导选择进行优化。该机制将有限算力在竞争性前缀间重新分配,而非固定单一展开路径,同时提供可调的粒子数量与可预测峰值内存。未来价值信号通过短程回溯模拟实现,也可替换为轻量级学习型选择头。总体上,APPS 改善了无训练解码中的精度-耗时权衡,进一步支持‘推理时幂近似可恢复通常归因于后训练的增益’的观点。
原文摘要 · Abstract (English)
A recurring pattern in "reasoning without training" is that base LLMs already assign non-trivial probability mass to correct multi-step solutions; the bottleneck is locating these modes efficiently at inference time. Power sampling provides a principled way to bias decoding toward such modes by targeting p_theta(x)^alpha with alpha > 1, but practical approximations must account for future-dependent correction factors that determine which prefixes remain promising. We introduce Auxiliary Particle Power Sampling (APPS), a blockwise particle algorithm for approximating the sequence-level power target with a bounded population of partial solutions. APPS propagates hypotheses in parallel using proposal-corrected power reweighting and refines their survival through future-value-guided selection at resampling boundaries. This redistributes finite compute across competing prefixes rather than committing to a single unfolding path, while providing a direct scaling knob in the particle count and predictable peak memory. We instantiate the future-value signal with short-horizon rollouts and also study an amortized variant that replaces rollouts with a lightweight learned selection head. AMore broadly, APPS improves the accuracy--runtime trade-off of training-free decoding, further supporting the view that inference-time power approximation can recover gains often attributed to post-training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。