arXiv:2607.15655cs.CLcs.LG2026-07

动态调整生成步数,让扩散语言模型更快更准地出结果。

Adaptive Multi-Step Lookahead Decoding for Diffusion Language Models

论文配图:Adaptive Multi-Step Lookahead Decoding for Diffusion Language Models
图 1 · 摘自论文原文
  • 根据候选词得分方差动态决定是否继续前瞻搜索
  • 在需要时自动扩展分支,避免无效深搜
  • 相比固定步数前瞻,兼顾速度与生成质量

掩码扩散语言模型(DLMs)通过迭代优化掩码词元实现并行文本生成,是自回归解码的有力替代方案。近期基于前瞻的解码方法通过预探未来状态来提升准确率与效率的平衡。然而,现有方法多依赖浅层单步前瞻,虽优化即时信息增益,但在长轨迹解码中表现不佳。我们发现,简单扩展为深层前瞻也无效,因固定深度展开带来额外计算,且无法适应不同中间状态。为此,本文提出AdaLook,一种自适应前瞻框架:根据候选得分方差动态判断是否继续推进,并在中间状态需进一步探索时触发分支扩展。该设计避免了不必要的深搜,同时允许从有信息量的状态重新启动前瞻。在多个基准和模型上的实验表明,AdaLook在准确率与解码步数之间取得更优平衡,优于现有单步前瞻方法。

原文摘要 · Abstract (English)

Masked diffusion language models (DLMs) enable parallel text generation by iteratively refining masked tokens, offering a promising alternative to autoregressive decoding. Recent lookahead-based decoding methods improve the accuracy--efficiency trade-off by exploring future decoding states before committing token updates. However, existing approaches mainly rely on shallow one-step lookahead, which optimizes immediate information gain but can be suboptimal for longer-horizon decoding trajectories. Meanwhile, we find that a naive extension for deeper lookahead is also ineffective, as fixed-depth rollout introduces additional computation and cannot adapt to heterogeneous intermediate decoding states. Thus, in this work, we propose AdaLook, an adaptive lookahead framework for DLM decoding. AdaLook dynamically determines whether to continue rollout based on candidate-score variance and further enables branch expansion when intermediate rollout states require additional exploration. This design avoids unnecessary deep rollout while allowing the decoder to re-trigger lookahead from informative intermediate states. Experiments on various benchmarks and models demonstrate that AdaLook achieves a better accuracy--decoding steps trade-off than existing one-step lookahead decoding methods.

扩散模型文本生成自适应解码

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。