通过融合最后查询注意力提升多模态缓存压缩效果
Last But Not Least: Boundary Attention CalibratiON for Multimodal KV Cache Compression
- 用最后查询注意力补充观察窗口，更准识别关键信息
- 在极端压缩下平均提升7.5%，最高达30.9%
- 适合作为插件用于各类多模态模型的缓存优化
多模态大语言模型（MLLMs）虽具备强视觉-语言推理能力，但长视觉上下文导致巨大的键值缓存（KV cache）和高解码延迟。现有压缩方法依赖观察窗口注意力进行稳定的重要程度估计，但该聚合方式可能稀释稀疏的关键证据，并在激进压缩下丢弃与答案相关令牌。我们发现最后查询注意力可作为恢复此类证据的互补信号，尽管其无关信号可能引入噪声。为此提出BACON，一种即插即用的方法，通过融合最后查询证据校准观察窗口注意力，同时利用层内一致性和层间持续性抑制噪声。在多种基准、模型、压缩预算和方法下，BACON在最激进预算下平均提升多模态KV缓存压缩7.5%，最高达30.9%。
原文摘要 · Abstract (English)
Multimodal Large Language Models (MLLMs) achieve strong vision-language reasoning but incur large KV caches and high decoding latency with long visual contexts. Existing compression methods rely on observation window attention for stable token importance estimation, yet this aggregation can dilute sparse critical evidence and discard answer-relevant tokens under aggressive compression. We identify last query attention as a complementary signal for recovering such evidence, though its irrelevant signals may introduce additional noise. We propose BACON, a plug-and-play method that calibrates observation window attention with last query evidence while suppressing noise through intra-layer coherence and inter-layer persistence. Across diverse benchmarks, models, budgets, and compression methods, BACON improves multimodal KV-cache compression by 7.5% on average under the most aggressive budget, with gains up to 30.9%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。