arXiv:2601.12904cs.CLcs.AI2026-01被引 3

通过融合检索片段信息,实现高效低耗的LLM生成。

From Prefix Cache to Fusion RAG Cache: Accelerating LLM Inference in Retrieval-Augmented Generation

  • 预处理时融合相关文本块信息,增强上下文
  • 仅重计算少于15%的关键词,提升70%准确率
  • 适合需快速生成且依赖外部知识的场景

检索增强生成通过引入外部知识降低大模型幻觉,但增加提示长度导致计算开销和首令牌时间(TTFT)上升。现有方法尝试复用检索块的预处理键值缓存以加速,但因缺乏跨块上下文信息,生成质量显著下降,缓存复用潜力未被充分释放。本文提出FusionRAG,一种新型推理框架,优化RAG的离线预处理与在线重处理阶段。预处理阶段将相关文本块信息嵌入每个块;在线重处理阶段,仅对模型关注的token重新计算键值缓存。实验表明,在相同重计算比例下,FusionRAG显著提升生成质量:重计算少于15%的token时,相比基线最高提升70%的归一化F1分数,且TTFT较全注意力机制降低2.66倍至9.39倍。

原文摘要 · Abstract (English)

Retrieval-Augmented Generation enhances Large Language Models by integrating external knowledge, which reduces hallucinations but increases prompt length. This increase leads to higher computational costs and longer Time to First Token (TTFT). To mitigate this issue, existing solutions aim to reuse the preprocessed KV cache of each retrieved chunk to accelerate RAG. However, the lack of cross-chunk contextual information leads to a significant drop in generation quality, leaving the potential benefits of KV cache reuse largely unfulfilled. The challenge lies in how to reuse the precomputed KV cache of chunks while preserving generation quality. We propose FusionRAG, a novel inference framework that optimizes both the preprocessing and reprocessing stages of RAG. In the offline preprocessing stage, we embed information from other related text chunks into each chunk, while in the online reprocessing stage, we recompute the KV cache for tokens that the model focuses on. As a result, we achieve a better trade-off between generation quality and efficiency. According to our experiments, FusionRAG significantly improves generation quality at the same recomputation ratio compared to previous state-of-the-art solutions. By recomputing fewer than 15% of the tokens, FusionRAG achieves up to 70% higher normalized F1 scores than baselines and reduces TTFT by 2.66x-9.39x compared to Full Attention.

RAG推理加速缓存优化LLM

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。