arXiv:2609.05821cs.CLcs.CV2026-09

让视觉语言模型复用缓存时更聪明,省算力还快得多。

CONDUIT: A Unified Residual-Stream Restoration Framework for KV Cache Reuse in Vision-Language Models

论文配图:CONDUIT: A Unified Residual-Stream Restoration Framework for KV Cache Reuse in Vision-Language Models
图 1 · 摘自论文原文
  • 用残差流重建思路统一单图多图缓存复用,不改模型结构。
  • 10%刷新预算下,平均达全前缀计算97%以上性能。
  • 适合需要快速响应的长文本视觉问答场景。

视觉语言模型常需对重复出现的视觉内容回答新问题,复用键值(KV)缓存可避免昂贵的视觉前缀重新编码。然而,当相同视觉内容出现在不同前缀下时,精确前缀复用会失效。选择性重计算可在小视觉标记预算下恢复质量,但仅在刷新正确过期标记时有效。原始注意力选择可能浪费预算在高注意力但价值范数代理分数低的标记上,或与查询无关的图像上。为此,我们提出CONDUIT,一种无需训练的刷新策略,将单图和多图复用统一为残差流重建。基于范数加权注意力,CONDUIT使用缓存键-查询注意力和可访问的预输出缓存值范数代理对缓存视觉标记排序,并在全局选择前应用经验性的图像级相关性增强。单图情况下系数为1,规则退化为图像内标记选择。该方法保留模型架构和权重,推理时仅增加一次查询条件评分步骤。在10%刷新预算下,CONDUIT在三个VLM主干网络上,五数据集平均达到全前缀计算的97.0%-99.5%,优于其他预算方法;在MMLongBench-Doc延迟子集上,仅使用全前缀13.5%的浮点运算量,实现2.99倍的首个标记生成速度提升。

原文摘要 · Abstract (English)

Vision-language models (VLMs) often answer new questions about recurring visual content, where reusing the key-value (KV) cache can avoid re-encoding expensive visual prefixes. Exact-prefix reuse, however, fails when the same visual content appears under a changed prefix. Selective recomputation can recover quality under a small visual-token budget, but only when the right stale tokens are refreshed. Raw-attention selection can waste budget on high-attention tokens with small value-norm proxy scores and on query-irrelevant images. To address these failure modes, we propose CONDUIT, a training-free refresh policy that unifies single- and multi-image reuse as residual-stream restoration. Building on norm-weighted attention, CONDUIT ranks cached visual tokens using cached-key query attention and an accessible pre-output cached-value-norm proxy, then applies empirical image-level relevance amplification before one global selection. With one image, the coefficient is one and the rule reduces to intra-image token selection. The method preserves model architecture and weights, adding only a single query-conditioned scoring pass at inference. At a 10% refresh budget, CONDUIT achieves 97.0-99.5% of the corresponding full-prefill five-dataset average across three VLM backbones and leads budgeted methods on average; on the MMLongBench-Doc latency subset, it uses 13.5% of full-prefill FLOPs and achieves a 2.99x time-to-first-token speedup.

视觉语言模型缓存复用推理优化高效生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。