arXiv:2503.13288cs.LGcs.AI2025-03被引 12

通过前瞻采样平衡推理中的探索与利用,提升大模型生成效率与质量。

$ϕ$-Decoding: Adaptive Foresight Sampling for Balanced Inference-Time Exploration and Exploitation

论文配图:$ϕ$-Decoding: Adaptive Foresight Sampling for Balanced Inference-Time Exploration and Exploitation
图 1 · 摘自论文原文
  • 基于前瞻采样构建价值估计,结合聚类逼近双分布以优化决策
  • 在7个基准上超越强基线,性能与效率双提升
  • 支持自适应计算分配,适用于多种大模型和算力规模

推理时优化将计算扩展至生成精细推理步骤以提升性能。尽管先前的搜索策略缓解了自回归生成的短视问题,但庞大的搜索空间导致过度探索和不足利用。为实现高效平衡以获取最优步骤,我们提出将解码策略建模为前瞻采样,利用模拟未来步骤来获得全局最优步骤估计。在此基础上,我们提出新型解码策略ϕ-Decoding。为实现精确且表达性强的步骤价值估计,ϕ-Decoding通过前瞻与聚类近似两种分布,从联合分布中采样以选择最优步骤进行利用。为支持自适应计算分配,我们设计了基于宽度和深度的剪枝策略,具备轻量级特性以提升推理效率。在七个基准上的大量实验表明,ϕ-Decoding在性能与效率方面均优于强基线。额外分析显示其在多种LLM上具有泛化能力,并可在广泛计算预算下实现可扩展性。代码将发布于https://github.com/xufangzhi/phi-Decoding,开源PyPI包即将上线。

原文摘要 · Abstract (English)

Inference-time optimization scales computation to derive deliberate reasoning steps for effective performance. While previous search-based strategies address the short-sightedness of auto-regressive generation, the vast search space leads to excessive exploration and insufficient exploitation. To strike an efficient balance to derive the optimal step, we frame the decoding strategy as foresight sampling, leveraging simulated future steps to obtain globally optimal step estimation. Built on it, we propose a novel decoding strategy, named $ϕ$-Decoding. To provide a precise and expressive estimation of step value, $ϕ$-Decoding approximates two distributions via foresight and clustering. Sampling from the joint distribution, the optimal steps can be selected for exploitation. To support adaptive computation allocation, we propose in-width and in-depth pruning strategies, featuring a light-weight solution to achieve inference efficiency. Extensive experiments across seven benchmarks show $ϕ$-Decoding outperforms strong baselines in both performance and efficiency. Additional analysis demonstrates its generalization across various LLMs and scalability across a wide range of computing budgets. The code will be released at https://github.com/xufangzhi/phi-Decoding, and the open-source PyPI package is coming soon.

大模型推理解码策略自适应计算

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。