arXiv:2606.23581cs.DCcs.AI2026-06

提出无需训练的统一多模态缓存机制,解决重复推理时的冗余计算问题。

Kamera: Unified Position-Invariant Multimodal KV Cache for Training-Free Reuse

论文配图:Kamera: Unified Position-Invariant Multimodal KV Cache for Training-Free Reuse
图 1 · 摘自论文原文
  • 设计位置无关的低秩条件补丁,修复跨块依赖丢失问题
  • 在多跳推理任务上恢复全精度准确率,仅需极小缓存开销
  • 适用于视频、UI截图等重复内容场景,尤其适合资源受限部署

多模态智能体在推理过程中反复查看相同视频帧、UI截图和渲染结果,但每次回溯都需重新编码,因前缀缓存仅支持固定起始位置复用。本文发现该重复计算可避免,并揭示了简单缓存复用所丢失的关键:块间交叉条件信息。这种损失具有不对称性:单跳召回可通过标准状态合并完全恢复,而多跳推理依赖的深层扩散残差则被忽略。现有无位置缓存方案无法解决此问题。为此提出一种无需训练的小型低秩条件补丁,与每块缓存一同存储。复用操作简化为两个步骤:任意目标位置的精确RoPE重旋转,以及补丁恢复跨块关联。该方法使重排序、滑动窗口存活和召回三类操作均只需一次运算,无需重编码。一个秩-m补丁即可在跨块绑定基准测试中恢复完整任务精度,在两种注意力结构和双页文档问答任务上表现优异,仅占极小的KV缓存空间,并在生产级SGLang内核中将重预填充缓存重建至bf16舍入误差范围内,覆盖六种模型。条件信号在冗余视觉与视频流中最强,因此本方案在多模态代理最耗算力的重复计算环节效果最为显著。

原文摘要 · Abstract (English)

Multimodal agents repeatedly re-examine the same video frames, UI screenshots, and rendered artifacts as their context window slides and reasoning iterates, yet every look-back re-encodes from scratch, because prefix caches serve reuse only at a fixed leading position. We show this recompute is avoidable, and identify exactly what naive KV reuse loses: the cross-chunk conditioning a chunk absorbs from its neighbours. This loss is asymmetric. The direct readout of a cached chunk is recovered exactly and for free by the standard state-merge. What remains is a diffuse, low-rank residue concentrated in deep layers, invisible to single-hop retrieval but precisely what multi-hop reasoning binds on. Blind reuse therefore leaves single-hop recall intact while halving multi-hop accuracy; this is the failure mode prior position-independent caches, designed for single-context or single-image reuse, do not address. We repair it with a small, training-free low-rank conditioning patch stored alongside each position-free chunk. Reuse reduces to one operator across MLA, GQA, and MHA: exact RoPE re-rotation to any target position, plus the patch that restores cross-chunk binding. This makes three window operations cheap: reorder (one patch serves every ordering of a cached set), sliding-window survival (surviving chunks relocate via rotation only, zero re-encode), and recall (an evicted chunk is rehydrated by its patch, never re-encoded). A rank-m patch recovers full task accuracy on cross-chunk-binding benchmarks, MM-NIAH across two attention families and two-page doc-QA, at a fraction of the KV footprint, and reconstructs re-prefill KV to within bf16 rounding in a production SGLang kernel across six backbones. The conditioning signal is strongest in redundant vision and video streams, making our solution most impactful where multimodal agents spend their recompute budget.

多模态缓存优化推理加速位置无关

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。