提升扩散语言模型生成速度与质量，通过智能重审和依赖感知的纠错机制。
Dependency-Aware Revocable Decoding for Efficient Diffusion Large Language Model Inference
- 将令牌分为掩码、候选和未掩码状态，分阶段验证并动态调整影响。
- 在12个基准上实现2.71倍加速，Flickr30K上CIDEr提升4.35点。
- 无需训练，适合追求高效高质文本/图文生成的开发者使用。
扩散大语言模型（dLLMs）通过迭代去噪并行解码多个词元，为自回归生成提供了有前景的替代方案。然而，增加解码并行度常导致生成质量下降，因早期错误会污染后续上下文。可撤销解码通过重新评估已解码词元并重掩码不可靠词元来缓解此问题，但现有方法忽略了不可靠词元也可能污染验证上下文本身。我们识别出这一失效模式，提出无训练的依赖感知可撤销解码（DARD）框架，将词元划分为掩码、候选和未掩码状态。DARD使用排除低可靠性词元的精选上下文验证候选词元，并自适应调节其对后续解码的影响。在3个开源dLLM上跨12个文本与多模态基准的实验表明，DARD在速度-质量权衡曲线上持续优于近期可撤销解码方法，在Flickr30K上实现2.71倍加速与4.35点的CIDEr提升。
原文摘要 · Abstract (English)
Diffusion large language models (dLLMs) offer a promising alternative to autoregressive generation by decoding multiple tokens in parallel through iterative denoising. However, increasing decoding parallelism often degrades generation quality, as early errors can contaminate later contexts. Revocable decoding mitigates this issue by re-evaluating decoded tokens and remasking unreliable ones, but existing methods overlook that unreliable tokens may also corrupt the verification context itself. We identify this failure mode and propose Dependency-Aware Revocable Decoding (DARD), a training-free framework that separates tokens into masked, candidate, and unmasked states. DARD verifies candidate tokens using a selective context that excludes less reliable tokens and adaptively regulates their influence on subsequent decoding. Experiments across 12 textual and multimodal benchmarks on 3 open-source dLLMs show that DARD consistently improves the speed-quality Pareto frontier over recent revocable decoding methods, achieving a 2.71$\times$ speedup and a 4.35-point CIDEr score gain over Saber on Flickr30K.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。