通过跳过早期层冗余计算，实现扩散语言模型高效推理。
ES-dLLM: Efficient Inference for Diffusion Large Language Models by Early-Skipping
- 基于中间表示变化度与置信度评估词重要性，提前跳过低贡献词。
- 在H200上达到最高308.51 TPS，提速达16.8倍且保持生成质量。
- 无需训练，适用于需快速生成的文本场景，如对话系统。
扩散大语言模型（dLLMs）因其能捕捉双向上下文并支持并行生成，正成为自回归模型的有前景替代方案。然而，其推理仍因每轮迭代需处理完整输入上下文而计算开销巨大。本文分析dLLM生成动态，发现关键、值和隐藏状态在连续迭代中变化微小。基于此，提出无需训练的推理加速框架ES-dLLM，通过估计词的重要性，在早期层跳过低贡献词。词重要性由中间张量变化率与前序迭代置信度决定。在LLaDA-8B和Dream-7B上的实验表明，ES-dLLM在NVIDIA H200上分别实现最高226.57和308.51 tokens per second（TPS）的吞吐量，相较原生实现提速5.6×至16.8×，较最优缓存方法提升1.85×，同时保持生成质量。
原文摘要 · Abstract (English)
Diffusion large language models (dLLMs) are emerging as a promising alternative to autoregressive models (ARMs) due to their ability to capture bidirectional context and the potential for parallel generation. Despite the advantages, dLLM inference remains computationally expensive as the full input context is processed at every iteration. In this work, we analyze the generation dynamics of dLLMs and find that intermediate representations, including key, value, and hidden states, change only subtly across successive iterations. Leveraging this insight, we propose \textbf{ES-dLLM}, a training-free inference acceleration framework for dLLM that reduces computation by skipping tokens in early layers based on the estimated importance. Token importance is computed with intermediate tensor variation and confidence scores of previous iterations. Experiments on LLaDA-8B and Dream-7B demonstrate that ES-dLLM achieves throughput of up to 226.57 and 308.51 tokens per second (TPS), respectively, on an NVIDIA H200 GPU, delivering 5.6$\times$ to 16.8$\times$ speedup over the vanilla implementation and up to 1.85$\times$ over the state-of-the-art caching method, while preserving generation quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。