arXiv:2609.06663cs.CLcs.AI2026-09

通过几何感知的互补度量,提升多模态大模型的键值缓存效率。

ECOKV: Geometry-Aware KV Cache Eviction via Complementary Diversity Metrics

论文配图:ECOKV: Geometry-Aware KV Cache Eviction via Complementary Diversity Metrics
图 1 · 摘自论文原文
  • 融合欧氏距离与余弦相似度,从不同角度捕捉标记多样性。
  • 在多种压缩比下达到当前最优性能,可减少观察窗口以释放更多缓存空间。
  • 适用于需高效推理的多模态大模型部署场景,尤其适合资源受限环境。

尽管多模态大语言模型(MLLMs)在各类任务中表现优异,其可扩展性仍受限于键值缓存(KV cache)带来的内存与计算开销。现有缓存淘汰方法通常结合余弦相似度多样性度量与重要性度量来选择性保留关键键值对。然而,余弦相似度涉及归一化,会丢失幅度信息,且由于隐藏表示的各向异性特性,常导致各层相似度值普遍偏高。本文提出ECOKV,系统分析现有多样性度量能力,提出一种几何感知的复合度量,联合使用欧氏距离与余弦相似度,从互补视角捕捉标记多样性。进一步利用这两项度量估计每个注意力头的冗余程度,实现多样性和重要性评分的自适应加权。实验表明,常见的近期标记保留窗口可显著缩小,从而将更多缓存容量用于信息丰富的标记,实现稳定提升。大量实验证明,ECOKV在多种压缩比下均达到最先进性能,并可无缝集成至现有缓存淘汰方法中。我们还分析了重要性与多样性间的关系,并考察了各层与注意力头间的冗余模式。

原文摘要 · Abstract (English)

Although multimodal Large Language Models (MLLMs) excel in diverse tasks, their scalability remains limited by the memory and computational overhead of KV cache storage. Recent KV cache eviction approaches incorporate a cosine similarity-based diversity metric with importance metrics to selectively retain critical key-value pairs. However, cosine similarity involves normalization that discards magnitude information, and it often yields uniformly high similarity values across layers due to the anisotropy property of hidden representations. In our study ECOKV, we rigorously deconstruct the capabilities of existing diversity metrics. Moving beyond simple measurement, we propose a geometry-aware composite metric that jointly leverages Euclidean distance and cosine similarity to capture token diversity from complementary perspectives. Furthermore, we use these two metrics to estimate the redundancy level of each attention head, allowing adaptive weighting between diversity and importance scores during token selection. Finally, we demonstrate that the observation window commonly employed to preserve recent tokens can be substantially reduced, thereby allocating more cache capacity to informative tokens and yielding consistent improvements. Extensive experiments demonstrate that ECOKV achieves state-of-the-art performance under various compression ratios and can be seamlessly integrated with existing KV cache eviction methods. We further analyze the relationship between importance and diversity, and examine redundancy patterns across layers and attention heads.

大模型优化缓存管理多模态

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。