无需训练，用隐藏信号实现长上下文缓存精准淘汰
VestigeKV: The NoPE-MLA KV Cache Carries Its Own Eviction Signal in a Vestigial Branch
- 利用无位置编码模型中残留的64维分支作为独立淘汰信号
- 在8k至65k上下文下检索准确率保持1.00，32倍压缩后仍达0.92
- 适合追求高精度长文本生成的开发者，无需修改模型结构
长时序KV缓存需在查询出现前进行压缩，但传统基于注意力观测的选择方法在无位置编码的多头线性注意力（NoPE MLA）模型上失效（针尖检索率0.00-0.33）。本文提出VestigeKV：在Kimi Linear模型中，利用训练后残留的64维解耦分支（原为RoPE残余）作为与查询无关的显著性信号。读取每行11%数据，将缓存分为两层：顶部m行保留在被关注层，其余精确迁移至GPU驻留归档区，由认证触发器按步访问。全程无需训练、量化或权重/内核修改。代价可忽略：8倍压缩下检索率达1.00，32倍下为0.92，与全行选择无差距。被关注层仅0.25KB（Kimi Linear每标记8.1KB），归档区位精确且驻留于GPU，支持主机卸载以释放显存。标准配置召回率128倍，准确率1.00。若Kimi K3采用类似缓存布局，该方法可能适用——但仅限实测模型。在带旋转的RoPE MLA上，相同操作降至0.08（纯淘汰0.42）；无旋转时显著性才存在（顶层目标占2.3-6.7%对比10.2-46.8%），且在RoPE下无法实现查询无关的精确合并。所有阈值冻结于数据前，论文附带20个归档判定与8条闭合路径。
原文摘要 · Abstract (English)
The problem. A long-lived KV cache must be compressed before the queries that will read it exist; selection by observed attention (H2O, SnapKV) collapses there (0.00-0.33 needle retrieval on a NoPE MLA model), because a token's importance has not yet been observed. The method. On Kimi Linear, VestigeKV evicts by a query-independent signal the cache already carries: the 64-dimensional decoupled branch, a vestige of RoPE that NoPE training repurposes into a salience channel. Reading 11% of each row, it partitions the cache: the top-m rows stay in the attended tier; every other row moves -- exactly, never deleted -- to a GPU-resident archive reachable per step by a certified trigger. No training, no quantization, no weight or kernel change. Cost. Nothing measurable: retrieval holds at 1.00 under 8x and 0.92 under 32x from 8k to 65k context, zero gap to full-row selection. The attended tier is 0.25 KB of Kimi Linear's 8.1 KB per-token cache at 32x; the archive stays bit-exact and GPU-resident, with host offload as the VRAM-reclaiming variant. The recall tier -- the standard configuration -- holds 128x at 1.00. Kimi K3 is reported to use a NoPE Gated-MLA variant; if its cache layout matches, the method plausibly extends there -- we make no claim beyond the measured model. NoPE exclusivity. The identical operator on a RoPE MLA collapses to 0.08 (plain eviction: 0.42); query-independent salience itself exists only without rotation (top-1 targets span 2.3-6.7% of tokens vs. 10.2-46.8%), and query-universal exact merging is provably impossible under RoPE. All thresholds were frozen before data; 20 archived verdicts and 8 closed routes accompany the paper.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。