提出新压缩框架，突破大模型上下文记忆瓶颈。
More Than a Quick Glance: Overcoming the Greedy Bias in KV-Cache Compression
- 按层累积压缩，用保护因子控制内存分配。
- 在128k上下文下准确率领先旧方法10%。
- 适合长文本任务与内存敏感部署场景。
尽管大型语言模型理论上可支持超长上下文窗口，但实际部署受限于键值（KV）缓存内存的线性增长。现有压缩策略通过剪枝缓解此问题，却牺牲语义召回率以换取内存效率。本文提出LASER-KV（层累积选择与精确LSH召回），在严格累积预算下探索KV压缩极限。不同于固定摘要长度的方法，采用基于保护因子n的分块累积策略，有效隔离压缩与滑动窗口效应。在Babilong基准测试中，以往压缩方法在多种长上下文任务上性能下降15-30%。LASER-KV保持稳定表现，在128k上下文下准确率最高提升10%。研究挑战了仅凭注意力分数即可表征词元价值的普遍假设。
原文摘要 · Abstract (English)
While Large Language Models (LLMs) can theoretically support extensive context windows, their actual deployment is constrained by the linear growth of Key-Value (KV) cache memory. Prevailing compression strategies mitigate this through various pruning mechanisms, yet trade-off semantic recall for memory efficiency. In this work, we present LASER-KV (Layer Accumulated Selection with Exact-LSH Recall), a framework designed to test the limits of KV compression under a strict accumulative budgeting policy. We deviate from the standard fixed summary size approach by implementing a block-wise accumulation strategy governed by a protection divisor (n). This allows us to isolate the effects of compression from sliding window artifacts. Our experiments on the Babilong benchmark reveal performance degradation in previous compression methods by 15-30% on various long context tasks. LASER-KV maintains stable performance, achieving superior accuracies by a margin of upto 10% at 128k. These findings challenge the prevailing assumption that attention scores alone are a sufficient proxy for token utility.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。