让扩散语言模型无需训练就能提速27倍，还保持生成质量。
Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding
- 用分块近似缓存机制实现双向扩散模型的键值缓存复用。
- 在多个基准上实现最高27.6倍吞吐提升，准确率损失极小。
- 适合追求高速非自回归生成且关注部署实用性的研究者。
基于扩散的大型语言模型（Diffusion LLMs）在非自回归文本生成中展现出并行解码潜力。然而，开源模型的实际推理速度常落后于自回归模型，原因在于缺乏键值（KV）缓存，以及并行解码时多标记同时生成导致的质量下降。为此，我们提出一种针对双向扩散模型的新型分块近似KV缓存机制，实现缓存复用且性能损失可忽略。此外，我们识别出并行解码质量下降的根本原因是条件独立性假设破坏了标记依赖关系。为此，我们设计了一种置信度感知的并行解码策略，仅对置信度超过阈值的标记进行解码，缓解依赖违反问题并维持生成质量。在LLaDA和Dream模型上的实验表明，跨多个大模型基准测试，吞吐量最高提升27.6倍，准确率损失微乎其微，显著缩小与自回归模型的差距，为扩散型语言模型的实际部署铺平道路。
原文摘要 · Abstract (English)
Diffusion-based large language models (Diffusion LLMs) have shown promise for non-autoregressive text generation with parallel decoding capabilities. However, the practical inference speed of open-sourced Diffusion LLMs often lags behind autoregressive models due to the lack of Key-Value (KV) Cache and quality degradation when decoding multiple tokens simultaneously. To bridge this gap, we introduce a novel block-wise approximate KV Cache mechanism tailored for bidirectional diffusion models, enabling cache reuse with negligible performance drop. Additionally, we identify the root cause of generation quality degradation in parallel decoding as the disruption of token dependencies under the conditional independence assumption. To address this, we propose a confidence-aware parallel decoding strategy that selectively decodes tokens exceeding a confidence threshold, mitigating dependency violations and maintaining generation quality. Experimental results on LLaDA and Dream models across multiple LLM benchmarks demonstrate up to \textbf{27.6$\times$ throughput} improvement with minimal accuracy loss, closing the performance gap with autoregressive models and paving the way for practical deployment of Diffusion LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。