通过缓存与混合精度，让扩散语言模型生成更快更省资源。
LaCache: Exact Caching and Precision-Adaptive Inference for Diffusion Large Language Models
- 缓存三类中间结果，跳过重复计算但输出不变。
- 纯推理加速达1.3倍，结合其他方法最高提速40.2倍。
- 适合追求高效生成的开发者和部署团队。
基于扩散的大型语言模型（DLLMs）通过半自回归解码实现文本并行生成。然而，现有方法存在严重的算子级冗余：在去噪步骤中重新计算整个序列，忽略了块内前缀和掩码后缀保持不变的事实。本文提出LaCache，一种无需训练的加速框架，通过无损缓存和混合精度缓解该冗余。具体地，LaCache采用无损状态记忆（LSM），缓存三类中间结果：(i) EmbedCache用于嵌入输出，(ii) RoPECache用于分词预注意力状态，(iii) FACache用于FlashAttention中的在线Softmax统计。这些缓存使模型可跳过未改变标记的冗余计算，且不改变输出。为进一步缓解内存带宽瓶颈，LaCache引入针对扩散过程步数依赖激活分布的分组FP8量化策略，应用于FFN层。实验表明，仅使用LaCache即可实现约1.3倍端到端加速；结合已有加速方法，最高可达40.2倍端到端加速，同时保持相当的任务准确性。
原文摘要 · Abstract (English)
Diffusion-based Large Language Models(DLLMs) enable parallel generation via Semi-Autoregressive (SAR) decoding in text generation. However, current methods suffer from severe operator-level redundancy: they recompute the entire sequence during denoising steps, ignoring that the prefix and masked suffix remain invariant within a block. We propose LaCache, a training-free acceleration framework that alleviates this redundancy through lossless caching and mixed precision. Specifically, LaCache employs Lossless State Memoization (LSM) by caching three types of intermediate results: (i) EmbedCache for embedding outputs, (ii) RoPECache for token-wise pre-attention states, and (iii) FACache for the online softmax statistics within FlashAttention. These caches allow the model to skip redundant computation on unchanged tokens without altering the output. To further alleviate memory-bandwidth bottlenecks, LaCache inegrates a per-group FP8 quantization strategy for FFN layers, tailored to step-dependent activation distributions across the diffusion process. Experiments demonstrate that LaCache alone achieves approximately 1.3X end-to-end speedup over vanilla DLLM. When combined with existing acceleration methods, LaCache reaches up to 40.2X end-to-end speedup while maintaining comparable task accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。