让扩散模型自己当裁判,实现无损加速生成。
Self Speculative Decoding for Diffusion Large Language Models
- 利用模型自身并行预测多位置,单次前向传播完成多步验证。
- 在LLaDA和Dream模型上实现最高3.46倍加速,输出与逐步解码一致。
- 无需额外模型,适合追求高效推理的扩散语言模型应用。
基于扩散的大型语言模型(dLLMs)作为自回归模型的有力替代,凭借双向注意力和并行生成范式展现出独特优势。然而,现有并行解码方法的生成结果偏离逐步解码,可能引入性能下降,限制其实际部署。为此,我们提出自推测解码(SSD),一种无损推理加速方法,利用dLLM自身同时充当推测起草者与验证者,无需额外模块。SSD引入自起草机制,模型在单次前向传播中生成多个位置的预测,并通过分层验证树进行验证。与传统需独立草稿模型的推测解码不同,SSD通过利用dLLM固有的多位置并行预测能力,消除模型冗余与内存开销。该自推测方法可使模型在单次前向传播中逐步验证并接受多个标记。实验表明,SSD在开源模型LLaDA和Dream上实现了最高3.46倍的速度提升,且输出与逐步解码完全一致。代码将公开于GitHub。
原文摘要 · Abstract (English)
Diffusion-based Large Language Models (dLLMs) have emerged as a competitive alternative to autoregressive models, offering unique advantages through bidirectional attention and parallel generation paradigms. However, the generation results of current parallel decoding methods deviate from stepwise decoding, introducing potential performance degradation, which limits their practical deployment. To address this problem, we propose \textbf{S}elf \textbf{S}peculative \textbf{D}ecoding (SSD), a lossless inference acceleration method that leverages the dLLM itself as both speculative decoding drafter and verifier without auxiliary modules. SSD introduces a self-drafting mechanism where the model generates predictions for multiple positions, then verifies them through hierarchical verification trees in a single forward pass. Unlike traditional speculative decoding that requires separate draft models, SSD eliminates model redundancy and memory overhead by exploiting the dLLM's inherent parallel prediction capability for multiple positions. This self-speculative approach allows the model to progressively verify and accept multiple tokens in a single forward pass. Our experiments demonstrate that SSD achieves up to 3.46$\times$ speedup while keeping the output identical to stepwise decoding on open source models such as LLaDA and Dream. Code will be made publicly available on GitHub.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。