用欧氏距离选关键记忆，让长文本推理更省内存且更准。
ManifoldKV: Training-Free KV Cache Compression via Euclidean Outlier Detection
- 基于键向量到中心点的欧氏距离排序，同时捕捉方向和幅度信息。
- 在4K-16K上下文中压缩20%仍保持95.7%准确率，优于传统方法。
- 无需训练、代码仅3行，适合所有主流模型，特别擅长多关键词检索。
长序列推理受限于随序列长度线性增长的键值缓存（KV-cache）内存；因此，高效压缩需可靠选择保留哪些历史标记。现有基于几何的淘汰方法通过计算键与全局中心点的余弦相似度打分，但余弦相似度忽略尺度信息，可能误删语义重要标记。本文提出ManifoldKV，一种无需训练的评分方法，通过欧氏距离衡量键向量相对于中心点的角向与径向偏离，更全面捕捉信息。在RULER基准上，ManifoldKV在4K-16K上下文、20%压缩率下达到95.7%准确率，性能媲美最优几何基线，且在余弦方法失效的两种场景中表现更优：一是在多键检索任务中，有效减少方向冲突，50%压缩率下3键NIAH任务准确率达92.4%，较KeyDiff提升15.4个百分点；二是在64K上下文时，为缓解全局中心点稀释问题，引入WindowedManifoldKV，在25%压缩率下恢复至84.3%准确率，相比全局L2提升49个百分点，较KeyDiff提升3.2个百分点。该方法仅需3行代码，跨4类架构无需调参即可生效。
原文摘要 · Abstract (English)
Long-context inference is constrained by KV-cache memory, which grows linearly with sequence length; KV-cache compression therefore hinges on reliably selecting which past tokens to retain. Most geometry-based eviction methods score keys by cosine similarity to a global centroid, but cosine is scale-invariant and can discard magnitude cues that distinguish semantically salient tokens. We propose ManifoldKV, a training-free scorer that ranks tokens by Euclidean distance to the key centroid, capturing both angular and radial deviations. On the RULER benchmark, ManifoldKV achieves 95.7% accuracy at 4K-16K contexts with 20% compression; matching the best geometric baseline while improving robustness in two regimes where cosine scoring fails. First, on multi-key retrieval, ManifoldKV reduces directional collisions, achieving 92.4% vs KeyDiff's 77.0% (+15.4 points) on 3-key NIAH at 50% compression. Second, to address dilution and performance collapse of global centroids at 64K context, we introduce WindowedManifoldKV, which restores accuracy to 84.3% at 25% compression, a 49-point recovery over global L2 and +3.2 points over KeyDiff. The method requires only 3 lines of code and works across 4 architectures without tuning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。