用扩散模型并行生成候选文本，实现超6倍无损推理加速
DFlash: Block Diffusion for Flash Speculative Decoding
- 用轻量块扩散模型一次性生成多个候选词，突破传统逐词生成限制
- 在多模型多任务上实现6倍以上无损加速，最高比EAGLE-3快2.5倍
- 适合追求低延迟、高吞吐的LLM部署场景，尤其对长文本生成有效
自回归大语言模型虽性能优异，但因解码过程固有的顺序性导致推理延迟高、GPU利用率低。推测解码通过快速草稿模型生成候选输出，并由目标模型并行验证，可缓解该瓶颈；然而现有方法仍依赖自回归草稿，仍为顺序生成，难以实现显著提速。扩散语言模型提供了一种并行生成的替代方案，但当前扩散模型性能通常低于自回归模型。本文提出DFlash，一种采用轻量级块扩散模型进行并行草稿生成的推测解码框架。通过单次前向传播生成草稿词元，并利用目标模型提取的上下文特征对草稿模型进行条件化，DFlash实现了高效且高质量的草稿生成，显著提升接受率。实验表明，DFlash在多种模型与任务上实现超过6倍的无损加速，相比最先进的推测解码方法EAGLE-3最高提升2.5倍。
原文摘要 · Abstract (English)
Autoregressive large language models (LLMs) deliver strong performance but require inherently sequential decoding, leading to high inference latency and poor GPU utilization. Speculative decoding mitigates this bottleneck by using a fast draft model whose outputs are verified in parallel by the target LLM; however, existing methods still rely on autoregressive drafting, which remains sequential and limits practical speedups. Diffusion LLMs offer a promising alternative by enabling parallel generation, but current diffusion models typically underperform compared with autoregressive models. In this paper, we introduce DFlash, a speculative decoding framework that employs a lightweight block diffusion model for parallel drafting. By generating draft tokens in a single forward pass and conditioning the draft model on context features extracted from the target model, DFlash enables efficient drafting with high-quality outputs and higher acceptance rates. Experiments show that DFlash achieves over 6x lossless acceleration across a range of models and tasks, delivering up to 2.5x higher speedup than the state-of-the-art speculative decoding method EAGLE-3.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。