提出单次前向传播完成长文本缓存压缩的新方法，兼顾速度与效果。
Still: Amortized KV Cache Compaction in a Single Forward Pass
- 用轻量级Perceiver模型一次性生成压缩后的键值对
- 在8k~128k上下文长度下，压缩比达200倍仍保持高精度
- 支持迭代应用，适合超长文本生成任务
KV缓存是长序列语言模型部署中的内存瓶颈。现有压缩方法或轻量但受限于子集选择，或表达能力强但需逐上下文优化。本文提出Still，一个针对冻结基模型训练一次的轻量级每层Perceiver，在单次前向传播中生成紧凑的键值对。在Qwen和Gemma模型上，Still在压缩比8×至200×、上下文长度8k至128k范围内，始终位于速度-质量前沿。在长期上下文评估集RULER上，性能超越最强基线8–22分。相同压缩缓存还可用于自由格式摘要，保留大部分全上下文收益，并在HELMET和LongBench摘要对比中胜出。由于压缩为前向传播，Still可迭代应用，进入传统逐上下文方法无法触及的长序列场景。结果表明，摊销机制使长上下文缓存压缩成为可能，而合成方式让极端压缩下的状态依然有效。
原文摘要 · Abstract (English)
The KV cache is the memory bottleneck of long-horizon language model deployment. Practically, a deployable compactor must be lightweight enough to call during inference, expressive enough to preserve context under constraint, and reusable across a trajectory. Existing compaction methods satisfy only part of this requirement: selection methods are lightweight but subset-bound, while synthesis methods are expressive but rely on per-context optimization. Here we introduce Still, a small per-layer Perceiver trained once against a frozen base model that produces compact keys and values in a single forward pass. On Qwen and Gemma models, Still occupies the favorable side of the speed--quality frontier across compression ratios from $8\times$ to $200\times$ and context lengths from $8$k to $128$k. On the long-context RULER grid, Still exceeds the strongest baseline by 8--22 points. The same compact cache also supports free-form summarization, preserving most of the full-context gain on HELMET and winning a pairwise LongBench summarization comparison against KV-Distill. Because compaction is a forward pass, Still can be applied iteratively, entering a long-horizon regime unavailable to per-context methods. We show that amortization makes long-context cache compaction tractable, and synthesis makes its compact state useful at extreme compression.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。