提出自适应缓存机制,让扩散语言模型推理快9倍且不丢质量。
dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Caching
- 利用提示词静态、响应部分动态的特性,按特征相似度更新缓存。
- 在LongBench-HotpotQA上减少9.1倍计算量,延迟逼近自回归模型。
- 无需训练,适配主流扩散语言模型,适合部署优化场景。
自回归模型长期主导大语言模型领域。近期出现的基于扩散的大语言模型(dLLMs)通过迭代去噪生成文本,展现出显著优势与潜力。然而,dLLMs存在高推理延迟问题。传统自回归加速技术(如键值缓存)因双向注意力机制不适用。本文观察到dLLM推理中提示词固定、响应部分动态,多数词元在相邻去噪步骤间保持稳定。基于此,提出无需训练的自适应缓存框架dLLM-Cache,结合长间隔提示缓存与基于特征相似度的部分响应更新。该设计可高效复用中间计算,不损失模型性能。在代表性dLLM(包括LLaDA 8B和Dream 7B)上的实验表明,dLLM-Cache在LongBench-HotpotQA上实现高达9.1倍的FLOPs减少,输出质量保持竞争力。值得注意的是,该方法使dLLM推理延迟在多数情况下接近自回归模型水平。代码已公开:https://github.com/maomaocun/dLLM-cache。
原文摘要 · Abstract (English)
Autoregressive Models (ARMs) have long dominated the landscape of Large Language Models. Recently, a new paradigm has emerged in the form of diffusion-based Large Language Models (dLLMs), which generate text by iteratively denoising masked segments. This approach has shown significant advantages and potential. However, dLLMs suffer from high inference latency. Traditional ARM acceleration techniques, such as Key-Value caching, are incompatible with dLLMs due to their bidirectional attention mechanism. To address this specific challenge, our work begins with a key observation that dLLM inference involves a static prompt and a partially dynamic response, where most tokens remain stable across adjacent denoising steps. Based on this, we propose dLLM-Cache, a training-free adaptive caching framework that combines long-interval prompt caching with partial response updates guided by feature similarity. This design enables efficient reuse of intermediate computations without compromising model performance. Extensive experiments on representative dLLMs, including LLaDA 8B and Dream 7B, show that dLLM-Cache achieves up to 9.1x FLOPs reduction on LongBench-HotpotQA while maintaining competitive output quality. Notably, our method brings dLLM inference latency close to that of ARMs under many settings. The code for this work is publicly available at: https://github.com/maomaocun/dLLM-cache.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。