提出DAF框架，让RAG推理更快更准。
Decoupled Attention Fusion: Accelerating RAG with Efficient KV Cache Reuse
- 分三阶段解耦注意力，恢复文档间关联性
- 长文本下速度比CacheBlend快2倍，比全重算快5.6倍
- 兼容Flash-Attention，适合部署在主流推理引擎
检索增强生成（RAG）能有效缓解大语言模型的幻觉问题，但在长上下文场景中存在首字延迟过高的问题。复用预计算的文档键值缓存可缓解此问题，但会引入分布不匹配，导致离线缓存缺乏文档间的注意力模式，影响推理连贯性。CacheBlend通过选择性注意力减少重计算，但在长文本下准确率严重下降。为此，我们提出解耦注意力融合（DAF）框架，在保持高精度的同时显著降低重计算开销。DAF将注意力过程分为三个集成阶段：重要令牌自注意力以恢复缺失的文档间注意力、问题-文档自注意力用于标准推理，以及状态融合将二者输出拼接生成最终隐藏状态。通过将这些操作解耦为密集模式，DAF天然兼容Flash-Attention内核，最大化硬件利用率，无需复杂注意力掩码。实验表明，在vLLM上，DAF相较CacheBlend最高提速2倍，相较全重计算提速5.6倍，且无准确率损失。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) effectively mitigates hallucinations in Large Language Models (LLMs) but suffers from prohibitive Time-To-First-Token (TTFT) latency in long-context scenarios. Reusing pre-computed document KV caches addresses this but introduces a distribution mismatch, where offline caches lack the inter-document attention patterns required for coherent reasoning. CacheBlend reduces recomputation via selective attention, but suffers severe accuracy degradation at longer contexts. To address these challenges, we propose Decoupled Attention Fusion (DAF), a framework that maintains high accuracy while significantly reducing recomputation overhead. DAF decouples the attention process into three integrated stages: important-token self-attention to restore missing inter-document attention, question-document self-attention for standard inference, and a state fusion that concatenates their outputs to synthesize the final hidden states. By decoupling these operations into dense patterns, DAF is natively compatible with Flash-Attention kernels, maximizing hardware utilization without requiring complex attention masks. Experiments show that DAF delivers up to 2 times speedup over CacheBlend and 5.6 times over full recomputation with vLLM on long-context benchmarks, without sacrificing accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。