通过重排与校准，实现大模型长文本推理的高效低秩缓存压缩。
ReCalKV: Low-Rank KV Cache Compression via Head Reordering and Offline Calibration
- 按注意力头相似性分组重排键，提升低秩近似精度。
- 离线校准值投影矩阵，保持上下文信息准确无须训练。
- 高压缩比下性能损失极小，适合长文本生成场景。
大型语言模型（LLMs）虽表现优异，但长上下文推理受限于键值（KV）缓存占用过多内存。因此，KV缓存压缩成为实现高效长文本推理的关键。现有低秩方法忽略键与值的不同作用及重要性差异，在高压缩比下导致性能显著下降。为此，我们提出ReCalKV，一种后训练阶段的低秩KV缓存压缩方法，针对键和值分别设计优化策略。对于键，提出逐头相似性感知重排（HSR），将结构相似的注意力头聚类分组，通过分组奇异值分解（SVD）实现更精确的低秩近似；对于值，提出离线值校准（OVC），利用校准数据在无需训练的情况下高效校准值投影矩阵，确保上下文信息准确表示。大量实验表明，ReCalKV持续优于现有低秩压缩方法，在高压缩比下仅带来极小性能损失。代码与模型将开源于：https://github.com/XIANGLONGYAN/ReCalKV。
原文摘要 · Abstract (English)
Large language models (LLMs) have demonstrated remarkable performance, but their long-context reasoning remains constrained by the excessive memory required for the Key-Value (KV) cache. This makes KV cache compression a critical step toward efficient long-context inference. Recent methods have explored low-rank techniques to reduce the hidden size of the KV cache. However, they neglect the distinct roles and varying importance of Keys and Values, leading to significant performance drops under high compression. To address this, we propose ReCalKV, a post-training low-rank KV cache compression approach with tailored strategies for Keys and Values. For Keys, we propose Head-wise Similarity aware Reordering (HSR), which clusters structurally similar heads into groups, enabling more accurate low-rank approximation via grouped SVD. For Values, we propose Offline Value Calibration (OVC), which efficiently calibrates the value projection matrix using calibration data without training, ensuring an accurate representation of contextual information. Extensive experiments show that ReCalKV consistently outperforms existing low-rank compression methods, achieving high compression ratios with minimal performance loss. The code and models will be available at:https://github.com/XIANGLONGYAN/ReCalKV.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。