arXiv:2602.02579cs.OScs.AI2026-02被引 5

针对长文本问答中缓存重用效率低的问题,提出按用户查询动态选择关键token重计算。

ProphetKV: User-Query-Driven Selective Recomputation for Efficient KV Cache Reuse in Retrieval-Augmented Generation

  • 根据查询相关性动态排序token,优先重算真正有用的上下文。
  • 仅用20%重计算量就保持96%-101%的完整精度。
  • 适合需要高效长文本检索增强生成的场景,如智能客服、知识问答。

长上下文检索增强生成(RAG)的预填充阶段受计算开销严重制约。现有方法通过组合检索文档的预计算KV缓存,并重新处理部分标记以恢复跨注意力。然而,我们发现当前标记选择标准存在根本性“挤出效应”:全局显著但与用户查询无关的标记占用了有限的重计算预算,排挤了真正回答问题所需的关键标记,导致推理准确率下降。本文提出ProphetKV,一种面向RAG场景的用户查询驱动的KV缓存重用方法。ProphetKV基于语义相关性动态优先处理标记,并采用双阶段重计算流程,将层间注意力指标融合为高价值集合。通过确保重计算预算集中于填补检索内容与用户查询之间的信息差距,ProphetKV以极小开销实现高保真注意力恢复。大量实验表明,ProphetKV在仅20%重计算比例下,仍可保持96%-101%的全预填充精度,相比SOTA方法(如CacheBlend、EPIC、KVShare),在RULER上提升8.8%-24.9%,在LongBench上提升18.6%-50.9%。

原文摘要 · Abstract (English)

The prefill stage of long-context Retrieval-Augmented Generation (RAG) is severely bottlenecked by computational overhead. To mitigate this, recent methods assemble pre-calculated KV caches of retrieved RAG documents (by a user query) and reprocess selected tokens to recover cross-attention between these pre-calculated KV caches. However, we identify a fundamental "crowding-out effect" in current token selection criteria: globally salient but user-query-irrelevant tokens saturate the limited recomputation budget, displacing the tokens truly essential for answering the user query and degrading inference accuracy. We propose ProphetKV, a user-query-driven KV Cache reuse method for RAG scenarios. ProphetKV dynamically prioritizes tokens based on their semantic relevance to the user query and employs a dual-stage recomputation pipeline to fuse layer-wise attention metrics into a high-utility set. By ensuring the recomputation budget is dedicated to bridging the informational gap between retrieved context and the user query, ProphetKV achieves high-fidelity attention recovery with minimal overhead. Our extensive evaluation results show that ProphetKV retains 96%-101% of full-prefill accuracy with only a 20% recomputation ratio, while achieving accuracy improvements of 8.8%-24.9% on RULER and 18.6%-50.9% on LongBench over the state-of-the-art approaches (e.g., CacheBlend, EPIC, and KVShare).

RAGKV缓存长文本生成推理优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。