提出高效查询感知的缓存融合方法，显著加速RAG推理
QCFuse: Query-Aware Cache Fusion via Compressed View for Efficient RAG Serving
- 用压缩视图和块锚探测实现查询感知的缓存复用
- 在匹配全预填充质量下，平均提速1.7倍
- 适合需要高效率RAG服务的生产系统
检索增强生成（RAG）通过引入外部证据提升大模型回答质量，但检索内容处理使预填充阶段成为主要服务开销。现有缓存融合方法在质量和效率间难以平衡：快速的无查询依赖或仅末层查询-上下文选择器可能遗漏相关证据，而完整视图的查询感知选择器需全层上下文信息，导致层间流水线阻塞。本文提出QCFuse，一种基于压缩视图的查询感知缓存融合方法。QCFuse利用块锚查询探测将用户查询状态编码为紧凑的每块锚点，并通过关键层分析识别需重新计算的标记，无需全层检查。我们在SGLang中实现QCFuse，对四个开源大模型在六个数据集上进行评估。结果表明，QCFuse达到全预填充级别的生成质量；在相同质量下，相比全预填充平均提速1.7倍，相比最强的质量保持基线ProphetKV提速1.5倍。
原文摘要 · Abstract (English)
Retrieval-augmented generation (RAG) improves large language model (LLM) answer quality by grounding generation in external evidence, but processing retrieved contexts makes the prefill stage a dominant serving cost. RAG cache fusion reduces this cost by reusing precomputed key-value (KV) caches for retrieved chunks and selectively recomputing tokens under the current prompt. Existing selectors, however, face a dilemma between quality and efficiency: fast query-agnostic or final-layer query-to-context selectors can miss request-relevant evidence, whereas full-view query-aware selectors require broad context and layer visibility before recomputation and therefore stall the layer-wise cache-fusion pipeline. We present QCFuse, a compressed-view query-aware selector for RAG cache fusion. QCFuse uses chunk-anchor query probing to condition user-query states on compact per-chunk anchors and critical-layer profiling to identify recomputation tokens without all-layer inspection. We implement QCFuse in SGLang and evaluate it on four open-weight LLMs across six datasets. QCFuse reaches full-prefill-level quality. At matched quality, QCFuse achieves an average prefill-time speedup of 1.7x over full prefill and 1.5x over ProphetKV, the strongest quality-preserving baseline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。