保护缓存结构比评分机制更重要，能显著提升长文本生成质量。
Protection Is (Nearly) All You Need: Structural Protection Dominates Scoring in Globally Capped KV Eviction
- 通过在边界保留10%缓存空间，有效防止关键信息丢失。
- 在缓存容量为256时，恢复了参考上限69%~90%的性能表现。
- 即使简化评分策略，只要保护结构，也能媲美传统最优算法。
我们在共享全局缓存限制的解码环境中研究了键值缓存（KV Cache）淘汰策略。七种策略（LRU、H2O、SnapKV、StreamingLLM、Ada-KV、QUEST、Random）均存在提示边界漏洞：缺乏结构保护时，在六个纯Transformer模型上性能骤降至F1≤0.064。在每个边界保留10%缓存空间后，可在七个LongBench模型上将缓存容量从2048降至256时，恢复69%~90%的基准上限性能（保留率13%）；十模型测试集覆盖68%~98%。注意力质量分析显示（Qwen2.5-3B, N=30），位置0的前缀信息占约75%注意力质量，其余边界词接近0.41倍均匀期望值，导致评分器虽保留关键节点但仍丢弃结构性重要令牌。加入保护后，简化评分隔离策略在K=32时与LRU等效（Δ=0.02）；在K=8时，注意力策略两两收敛，且在C=256和C=512下比LRU提升0.011~0.021 F1。忠实的Ada-KV/QUEST在Mistral-7B和Phi-3.5上额外带来0.03~0.04 F1增益。对Qwen3-4B的NIAH-32K迁移实验（解码与预填充，C∈{512,2048}）表明，保护带来的提升比例为0.99~1.00。在64K场景下，保护仍有效但恢复有限；仅当模型本身支持强64K检索而无需淘汰时，全缓存天花板性能才在6.3%保留率下达成。总体结论：结构保护主导性能表现，评分差异在边界受保护后退居次要，每头分配带来小幅增益。
原文摘要 · Abstract (English)
We study KV cache eviction under a shared globally capped decode-time harness. Seven policies (LRU, H2O, SnapKV, StreamingLLM, Ada-KV, QUEST, Random) share a prompt-boundary vulnerability: without structural protection, they collapse to near-zero quality on six pure-transformer models (F1$\leq$0.064). Reserving 10\% of cache at each boundary recovers 69--90\% of the $C{=}2{,}048$ reference-ceiling quality on seven LongBench models at $C{=}256$ (13\% retention); a ten-model panel spans 68--98\%. An attention-mass pilot (Qwen2.5-3B, $N{=}30$) suggests why: the position-0 sink holds ${\sim}75\%$ of prefix mass, while other boundary tokens sit near ${\sim}0.41{\times}$ uniform expectation, so attention scorers retain the sink but still drop structurally critical tokens. With protection, simplified score-isolation variants are TOST-equivalent to LRU at $K{=}32$ ($Δ{=}0.02$); at $K{=}8$, attention policies pairwise converge yet beat LRU by 0.011--0.021 F1 across $C{=}256$ and $C{=}512$. Faithful Ada-KV/QUEST add ${\sim}0.03$--$0.04$ F1 on Mistral-7B and Phi-3.5 beyond simplified variants. A NIAH-32K regime-transfer pilot on Qwen3-4B (decode vs.\ prefill, $C{\in}\{512,2048\}$) shows near-identical protection lifts (ratio 0.99--1.00). At 64K, protection helps but recovery is modest; faithful per-head scoring matches full-cache ceiling on Gemma-3-4B at 6.3\% retention only when the model already supports strong 64K retrieval without eviction. Overall: protection dominates; scoring differences are secondary once boundaries are guarded; per-head allocation gives a further modest gain.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。