arXiv:2607.11942cs.LGcs.AI2026-07

测试了不同压缩方法在有无查询信息时的表现,发现查询可见性会改变排名。

How Query Visibility Changes KV-Cache Compression Rankings: A Matched-Budget Audit

  • 对比了六种压缩方法在有无查询信息下的表现,固定其他条件只变评分规则。
  • 无查询时,只有KeyDiff持续优于基线;最常用方法SnapKV反而更差。
  • 方法得分对查询的依赖程度越高,性能下降越明显,可从代码中看出。

KV缓存压缩方法通常在压缩前将查询附加到上下文中进行评估——这是一种查询感知协议。然而,压缩的实际经济价值在于复用:压缩一次文档,可回答多次后续问题。在这种部署场景中,压缩必须在任何查询出现前完成,即查询无关。本文对六种已发表的压缩方法与三种简单基线,在三个7-9B参数规模的开源模型上进行了匹配预算审计(在RULER-8192上共144,300次配对评估;LongBench上40,800次;50,000次重抽样自举检验)。所有因素均保持一致——模型、压缩比、样本实例、解码过程——仅评分规则变化。三项发现:(1) 查询可见性改变排名:在无关协议下,五个共享相同注意力后端的方法中,仅有KeyDiff在36个单元中的31个上持续优于最佳基线,而最广泛使用的SnapKV平均表现低于‘保留开头和最近窗口’基线(-0.066);(2) 各方法在两种协议间的性能下降幅度,与其评分信号对查询的可见性呈一致顺序:从SnapKV的Δ=+0.198(查询在其64令牌观测窗口内)到KeyDiff的Δ=+0.011(其评分不包含任何查询词)。

原文摘要 · Abstract (English)

KV-cache compression methods are predominantly evaluated with the query appended to the context before compression -- a query-aware protocol. Yet the economic case for a compressed KV cache is reuse: compress a document once, answer many future questions against it. In that deployment, compression must happen query-agnostic -- before any question is seen. We present a matched-budget audit of six published compression methods against three trivial baselines on three open 7-9B models (144,300 paired evaluations on RULER-8192; 40,800 on LongBench; 50,000-resample paired bootstrap throughout). Everything is held fixed -- model, compression ratio, instances, decoding -- except the scoring rule. Three findings. (1) Query visibility changes the rankings: under the agnostic protocol, of the five audited methods that share a common attention backend, only KeyDiff beats a best-of-3 trivial baseline consistently (31 of 36 cells), and the most widely deployed method, SnapKV, loses to "keep the start and the recent window" on average (-0.066). (2) The per-method drop between the two protocols is ordered consistently with how visible the question is to each method's scoring signal, legible in its source code: from Delta=+0.198 for SnapKV (the question sits inside its 64-token observation window) down to Delta=+0.011 for KeyDiff (its score contains no query term at all).

KV缓存压缩评测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。