提出新缓存机制，让扩散型大模型推理更快更准
Affix Cache for Diffusion Large Language Models
- 识别关键前缀词，只重算部分缓存，其余复用
- 重算约20%关键词，准确率恢复至原水平
- 适用于需要快速生成的文本任务，提升推理效率
扩散型大语言模型（DLLMs）支持非自回归解码和双向上下文建模，但高效推理仍具挑战。与自回归系统可复用共享前缀的键值（KV）缓存不同，DLLMs通过双向注意力将共享上下文词元的KV状态与生成词元耦合，导致直接缓存复用失效，而全量重算又成本高昂。本文提出ACache，一种面向共享文本片段（不限于前缀）的缓存复用机制。ACache通过测量词元对掩码生成词元的影响，识别出少量请求相关的关键词元（锚点词元），仅对这些词元重算其KV状态，其余部分复用原有缓存。基于Fast-dLLM实现的ACache，在不同设置下仅重算约20%的词元即可恢复直接复用造成的精度损失。此外，基于Nano-vLLM构建的共享前缀原型表明，ACache将重算延迟降低最多55.7%，端到端吞吐提升最高1.68倍。
原文摘要 · Abstract (English)
Diffusion Large Language Models (DLLMs) enable non-autoregressive decoding and bidirectional context modeling, but efficient inference remains challenging. Unlike autoregressive systems, whose key-value (KV) cache can be reused for shared prefixes, DLLMs couple the KV states of shared context tokens with evolving generated tokens through bidirectional attention, making naive cache reuse stale while full recomputation is expensive. We present ACache, an affix-oriented cache reuse mechanism for shared text spans in DLLMs beyond prefixes. ACache identifies a small request-specific subset of critical affix tokens, called Anchor Tokens, by measuring their influence on masked generation tokens, and selectively recomputes the KV states of only these tokens while reusing the remaining affix cache. Built on Fast-dLLM, ACache recovers the accuracy loss caused by direct affix-cache reuse across different settings when recomputing around 20% of affix tokens. We also build a shared-prefix prototype on top of the Nano-vLLM engine, showing that ACache reduces recompute latency by up to 55.7% and improves end-to-end throughput by up to 1.68$\times$.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。