用图像缓存提升RAG效率，同时保持文本精度
QV-PIC: Query-Aware Visual Position-Independent Caching for Efficient RAG Serving
- 将文本转为图像缓存，用双分辨率策略复用计算结果
- 在六项任务中提升平均准确率21.6点，减少首次响应时间83.8%
- 适合需要高效推理且对内容精度要求高的大模型应用
检索增强生成（RAG）在多个查询中重复预填充相同文本块，造成冗余计算。位置无关缓存（PIC）通过跨位置复用预计算的键值对缓解此问题，但受限于文本令牌数量庞大。将文本块渲染为图像可压缩为更少视觉令牌，但图像版PIC质量下降更严重。这一表现差异主要源于独立缓存间的上下文不一致，以及视觉压缩中丢失细粒度文本证据。现有修复方法多通过选择性重计算解决前者，但需在线计算且无法恢复丢失的文本细节。本文提出QV-PIC，一种由模型原生模板引导的查询感知双分辨率缓存复用框架。离线阶段，基于模型原生聊天模板编译视觉缓存，提升质量且无需在线重计算；在线阶段，以低分辨率保留全局上下文，并通过累积查询相关性分数，在高分辨率预算内恢复细粒度文本证据，兼顾视觉压缩效率。在六个任务上，QV-PIC相较原始图像版PIC平均提升F1 21.6点，缩小与文本版PIC差距，优于优化版文本PIC 2.58 F1，TTFT降低17.2%。相比完整预填充，TTFT降低83.8%。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) repeatedly prefills identical text chunks across queries, incurring redundant computations. Position-Independent Caching (PIC) mitigates it by reusing precomputed Key-Value (KV) across positions, but its efficiency is constrained by the large volume of text tokens. Rendering text chunks as images can compress the text into fewer visual tokens, but the rendered-image PIC suffers more severe quality degradation than the text PIC. This representation-specific gap primarily arises from contextual mismatches across independently compiled caches and the loss of fine-grained textual evidence during visual compression. Existing PIC repair methods mainly address the former through selective recomputation, but they incur online computation and cannot recover lost textual details. We propose QV-PIC, a query-aware dual-resolution PIC reuse framework guided by model-native templates. Offline, QV-PIC compiles visual caches under the model's native chat-template prefix, improving PIC quality without online recomputation. Online, it preserves global context with low resolution and restores fine-grained textual evidence within a high-resolution budget by cumulative query relevance scores, retaining the efficiency benefit of visual compression. Across six tasks, QV-PIC improves average F1 by 21.6 points over vanilla rendered-image PIC, closes the gap to vanilla text PIC, and surpasses optimized text PIC by 2.58 F1 while reducing TTFT by 17.2\%. Relative to full prefill, it cuts TTFT by 83.8%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。