通过信息流信号优化长文本问答中的关键值缓存重计算
InfoFlow KV: Information-Flow-Aware KV Recomputation for Long Context
- 用查询注意力归一化信号判断哪些词能有效传递信息
- 在保持推理延迟不变的前提下，显著提升长上下文生成效果
- 适合需要高效处理超长文档的问答系统开发者
长上下文问答中的检索增强生成（RAG）受限于大段检索内容的推理时预填充。常见做法是预先计算各文档的键值（KV）缓存，并选择性重计算部分标记以恢复全局因果关系，但现有方法依赖启发式规则或表示差异，未建模所选标记是否真正影响生成。本文将选择性KV重计算视为信息流问题，发现查询注意力归一化信号在一致的RoPE几何下可可靠识别出语义相关且结构上利于信息传播的标记。因此，我们重构检索块的全局位置分配，并提出基于信息流引导的块重排序策略。在大语言模型与视觉-语言模型基准上的实验表明，在相近延迟条件下，该方法持续优于现有方法。
原文摘要 · Abstract (English)
Retrieval-augmented generation (RAG) for long-context question answering is bottlenecked by inference-time prefilling over large retrieved contexts. A common strategy is to precompute key-value (KV) caches for individual documents and selectively recompute a small subset of tokens to restore global causal dependencies, but existing methods rely on heuristics or representation discrepancies without modeling whether selected tokens can effectively influence generation. We cast selective KV recomputation as an information flow problem and show that a simple attention-norm signal from the query reliably identifies tokens that are both semantically relevant and structurally positioned to propagate information, when computed under an inference-consistent RoPE geometry. We therefore reconstruct global positional assignments for retrieved chunks and introduce an information-flow-guided chunk reordering strategy. Experiments on Large Language Model and Vision-Language Model benchmarks demonstrate consistent gains over prior methods under comparable latency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。