arXiv:2604.08964cs.CL2026-04ACL被引 3

提出AHD方法,让扩散语言模型解码更高效准确。

Breaking Block Boundaries: Anchor-based History-stable Decoding for Diffusion Large Language Models

  • 用动态锚点实时监测词元稳定性,提前释放跨块生成
  • 在BBH上减少80%解码步数,性能提升3.67%
  • 无需训练,可直接插入现有模型,适合加速推理

扩散大语言模型(dLLMs)正成为自回归模型的有力替代。半自回归(Semi-AR)解码因其优异性能被广泛用于基础dLLMs及先进解码策略中。然而我们发现,Semi-AR解码存在固有的块边界约束,导致大量跨块稳定词元的生成被不必要的延迟。针对此问题,我们系统研究了稳定词元的识别机制,得出三项关键发现:(1) 直接前瞻解码不可靠,(2) 词元稳定性与收敛趋势密切相关,(3) 历史信息相互隔离。基于这些洞察,我们提出无需训练、即插即用的锚点历史稳定解码(AHD)。AHD通过动态锚点实时监控词元稳定性趋势,一旦确认稳定即触发跨块提前解码,显著提升效率与性能。在语言、视觉-语言和音频-语言等多个领域实验表明,AHD同时提升性能与推理效率。尤其在BBH基准上,解码步数减少80%,性能提升3.67%,有效逆转了现有加速策略常见的性能下降现象。

原文摘要 · Abstract (English)

Diffusion Large Language Models (dLLMs) have recently become a promising alternative to autoregressive large language models (ARMs). Semi-autoregressive (Semi-AR) decoding is widely employed in base dLLMs and advanced decoding strategies due to its superior performance. However, our observations reveal that Semi-AR decoding suffers from inherent block constraints, which cause the decoding of many cross-block stable tokens to be unnecessarily delayed. To address this challenge, we systematically investigate the identification of stable tokens and present three key findings: (1) naive lookahead decoding is unreliable, (2) token stability closely correlates with convergence trend, and (3) historical information is isolated. Building on these insights, we propose Anchor-based History-stable Decoding (AHD), a training-free, plug-and-play dynamic decoding strategy. Specifically, AHD monitors the stability trend of tokens in real time through dynamic anchors. Once a token reaches stability, it initiates early cross-block decoding to enhance efficiency and performance. Extensive experiments across language, vision-language, and audio-language domains demonstrate that AHD simultaneously improves both performance and inference efficiency. Notably, AHD effectively reverses the performance degradation typically observed in existing advanced decoding acceleration strategies. For instance, on the BBH benchmark, our approach reduces decoding steps by 80% while improving performance by 3.67%.

扩散模型解码优化推理加速

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。