通过多时间尺度异常检测压缩长文本记忆，显著提升小缓存下的模型表现。
NestedKV: Nested Memory Routing for Long-Context KV Cache Compression
- 用全局、块级、滑动窗口三类锚点捕捉不同粒度的上下文重要性。
- 在小缓存比例下（r=0.75），RULER和LongBench得分比现有方法高19分以上。
- 无需训练，适配多模型，特别适合资源受限场景下的长文本推理。
长上下文语言模型受限于键值（KV）缓存的内存开销。现有无训练的KV压缩方法通常仅依赖单一重要性信号——注意力、新近性、分层分配或键的独特性——当有用上下文呈现全局独特、局部片段化或即时相关时，这些方法易失效。本文提出NestedKV，一种受嵌套学习中连续记忆系统启发的仅键压缩方法。它维护全局、块级和滑动窗口三类键锚点，通过多时间尺度余弦异常分数对令牌排序，并结合无训练外部学习器，采用头自适应混合与惊喜门控路由策略融合排名结果。该评分机制配合每头自适应预算，无需训练或修改大模型。在Qwen3与Llama-3.2上，针对RULER（4k–32k）、LooGLE、LongBench、LongBench-E、InfiniteBench及MMLU-Pro数据集测试表明，当保留缓存比例较小时，NestedKV表现最优。在Qwen3-4B模型上，于r=0.75时，相较于KeyDiff，RULER最高提升19.10分，LongBench提升19.29分；在r=0.95时，LongBench得分达37.32，远超KeyDiff的17.55。
原文摘要 · Abstract (English)
Long-context language models are limited by the memory footprint of the key-value (KV) cache. Existing training-free KV compression methods usually rank tokens by one importance signal -- attention, recency, layer-wise allocation, or key distinctiveness -- which becomes brittle when useful context is globally distinctive, locally episodic, or immediately relevant. We introduce NestedKV, a key-only KV cache compression method inspired by the Continuum Memory System in Nested Learning. NestedKV maintains global, block-level, and sliding-window key anchors, scores tokens by multi-time-scale cosine anomaly, and combines the resulting rankings with a training-free outer learner using head-adaptive mixing and surprise-gated token routing. The score is paired with adaptive per-head budgets and requires no training or LLM modification. Across RULER (4k--32k), LooGLE, LongBench, LongBench-E, InfiniteBench, and MMLU-Pro on Qwen3 and Llama-3.2 models, NestedKV is strongest when the retained cache is small. On Qwen3-4B, it improves over KeyDiff by up to 19.10 points on RULER and 19.29 on LongBench at $r=0.75$; at $r=0.95$, it retains 37.32 on LongBench versus 17.55 for KeyDiff.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。