arXiv:2602.05929cs.CL2026-02被引 3

提出新方法评估大模型键值缓存的压缩潜力,发现压缩性与模型架构和语言有关。

KV-CoRE: Benchmarking Data-Dependent Low-Rank Compressibility of KV-Caches in LLMs

  • 基于SVD的无梯度增量方法,量化键值缓存的低秩可压缩性。
  • 在五种英语领域、十六种语言上验证,压缩性与模型架构和训练数据强相关。
  • 提供首个大规模基准,适合研究动态压缩和数据驱动模型设计的人参考。

大型语言模型依赖键值缓存(KV-cache)避免自回归解码中的重复计算,但随着上下文长度增加,缓存的读写会迅速耗尽GPU内存带宽。尽管已有研究探索缓存压缩,但多数方法忽略了缓存的数据依赖性及其在各层间的差异。本文提出KV-CoRE(KV-cache Compressibility by Rank Evaluation),一种基于SVD的方法,用于量化数据依赖的低秩压缩能力。该方法在弗罗贝尼乌斯范数下计算最优低秩近似,具有无梯度、可增量计算的特点,支持在数据集级别和层间高效评估。我们对多个模型和数据集进行了分析,涵盖五个英语领域和十六种语言,揭示了压缩性与模型架构、训练数据及语言覆盖之间的系统性规律。通过引入归一化有效秩作为压缩性度量,发现其与压缩下的性能退化高度相关。本研究建立了原理性的评估框架,首次构建了大规模的键值缓存压缩性基准,为动态、数据感知的压缩策略和数据导向的模型开发提供了洞见。

原文摘要 · Abstract (English)

Large language models rely on kv-caches to avoid redundant computation during autoregressive decoding, but as context length grows, reading and writing the cache can quickly saturate GPU memory bandwidth. Recent work has explored KV-cache compression, yet most approaches neglect the data-dependent nature of kv-caches and their variation across layers. We introduce KV-CoRE KV-cache Compressibility by Rank Evaluation), an SVD-based method for quantifying the data-dependent low-rank compressibility of kv-caches. KV-CoRE computes the optimal low-rank approximation under the Frobenius norm and, being gradient-free and incremental, enables efficient dataset-level, layer-wise evaluation. Using this method, we analyze multiple models and datasets spanning five English domains and sixteen languages, uncovering systematic patterns that link compressibility to model architecture, training data, and language coverage. As part of this analysis, we employ the Normalized Effective Rank as a metric of compressibility and show that it correlates strongly with performance degradation under compression. Our study establishes a principled evaluation framework and the first large-scale benchmark of kv-cache compressibility in LLMs, offering insights for dynamic, data-aware compression and data-centric model development.

大模型缓存压缩低秩分解评估基准

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。