用概率理论改进大模型推理,让每一步决策更长远、更高效。
Martingale Foresight Sampling: A Principled Approach to Inference-Time LLM Decoding
- 将推理路径建模为随机过程,用数学理论替代经验规则。
- 在6个基准上准确率超越现有方法,计算开销更低。
- 适合追求推理质量与效率的AI研究者和工程师。
大型语言模型(LLM)的标准自回归解码存在短视问题,难以找到全局最优的推理路径。尽管推理时的前瞻采样等策略尝试通过模拟未来步骤缓解此问题,但通常依赖经验性启发式方法来评估路径并剪枝搜索空间。本文提出马丁格尔前瞻采样(MFS),将解码重构为寻找最优随机过程的问题。通过将推理路径质量建模为随机过程,利用鞅理论设计理论严谨的算法:基于多布分解定理计算每步的可预测优势,运用可选停止理论实现对次优路径的合理剪枝,依据鞅收敛定理设计自适应终止规则,在路径质量已收敛时停止探索。在六个推理基准上的实验表明,MFS在准确率上超越当前最佳方法,并显著提升计算效率。代码将于 https://github.com/miraclehetech/EACL2026-Martingale-Foresight-Sampling 公开。
原文摘要 · Abstract (English)
Standard autoregressive decoding in large language models (LLMs) is inherently short-sighted, often failing to find globally optimal reasoning paths due to its token-by-token generation process. While inference-time strategies like foresight sampling attempt to mitigate this by simulating future steps, they typically rely on ad-hoc heuristics for valuing paths and pruning the search space. This paper introduces Martingale Foresight Sampling (MFS), a principled framework that reformulates LLM decoding as a problem of identifying an optimal stochastic process. By modeling the quality of a reasoning path as a stochastic process, we leverage Martingale theory to design a theoretically-grounded algorithm. Our approach replaces heuristic mechanisms with principles from probability theory: step valuation is derived from the Doob Decomposition Theorem to measure a path's predictable advantage, path selection uses Optional Stopping Theory for principled pruning of suboptimal candidates, and an adaptive stopping rule based on the Martingale Convergence Theorem terminates exploration once a path's quality has provably converged. Experiments on six reasoning benchmarks demonstrate that MFS surpasses state-of-the-art methods in accuracy while significantly improving computational efficiency. Code will be released at https://github.com/miraclehetech/EACL2026-Martingale-Foresight-Sampling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。