通过跨模态信息流优化多模态缓存合并，显著降低内存与延迟。
FlowMM: Cross-Modal Information Flow Guided KV Cache Merging for Efficient Multimodal Context Inference
- 基于跨模态信息流动态调整分层合并策略，适应不同模态特征。
- 实现80%-95%缓存内存减少，推理延迟降低1.3-1.8倍。
- 适合高并发多模态大模型应用，如视频生成与图文理解。
传统KV缓存淘汰策略依据注意力得分丢弃低重要性键值对，常导致生成质量下降、上下文丢失或幻觉。近期研究转向KV合并，通过相似性将待淘汰与保留的键值对合并。但在多模态场景中，不同模态间分布偏差及跨模态注意力偏差限制了其效果。本文提出FlowMM，一种基于跨模态信息流引导的自适应多模态KV缓存合并框架。FlowMM利用跨模态信息流动态应用分层合并策略，捕捉模态特异性模式并保持上下文完整性。同时引入敏感度自适应的标记匹配机制，联合评估标记相似性与任务敏感度，仅合并低风险项而保护高敏感项。在多个主流多模态大模型上的实验表明，FlowMM可实现80%至95%的KV缓存内存压缩，解码延迟降低1.3至1.8倍，同时保持竞争力的任务性能。
原文摘要 · Abstract (English)
Traditional KV cache eviction strategies, which discard less critical KV-pairs based on attention scores, often degrade generation quality, causing context loss or hallucinations. Recent efforts shift toward KV merging, merging eviction tokens with retention tokens based on similarity. However, in multimodal scenarios, distributional biases across modality tokens and attentional biases in cross-modal interactions limit its effectiveness. This work introduces FlowMM, an adaptive framework for cross-modal information flow-guided multimodal KV cache merging. FlowMM leverages cross-modal information flow to dynamically apply layer-specific merging strategies, capturing modality-specific patterns while preserving contextual integrity. Furthermore, we introduce a sensitivity-adaptive token matching mechanism that jointly evaluates token similarity and task-critical sensitivity, merging low-risk tokens while safeguarding high-sensitivity ones. Extensive experiments across diverse leading MLLMs show that FlowMM reduces KV cache memory by 80% to 95% and decoding latency by 1.3-1.8x, while maintaining competitive task performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。