arXiv:2607.01520cs.LG2026-07被引 3

揭示长序列推理中KV缓存压缩的理论极限,指导高效压缩算法设计。

The risk of KV cache compression

  • 基于缓存内在可压缩性,建立压缩风险的理论框架。
  • 在长序列任务上,新算法实现最小化最大误差,优于现有方法。
  • 适合追求理论保障的模型部署与推理优化研究者。

Transformer 在长序列推理中成本高昂,因 softmax 注意力需反复读取庞大的 KV 缓存。主流解决方案是通过紧凑摘要替代完整缓存来压缩 KV 缓存。尽管该技术具有重要实践意义,但摘要设计主要依赖经验试错。理论上,已有研究指出在最坏情况下压缩可能不可行,却未能为可行压缩场景提供系统性设计指导。本文通过量化缓存压缩的极小极大风险,揭示了准确压缩的可行性条件。由此导出的新设计原则适用于因果掩码场景,能高效映射至预填充和自回归解码阶段,并达到极小极大最优风险。我们在实践中实现该原则,针对 LongBench 进行实验,验证了其优异性能。整体结果为具备理论保证的实用化缓存压缩提供了坚实路径。

原文摘要 · Abstract (English)

Transformer inference on long sequences is expensive because softmax attention repeatedly reads from a large KV cache. The prevalent approach to this bottleneck is KV cache compression, which replaces the full cache with a compact summary. Despite its practical importance, the design of such summaries is largely driven by empirical experimentation. On the theoretical side, existing results show that KV cache compression can be impossible in the worst case, but offer little systematic guidance for designing algorithms in regimes where accurate compression is possible. We bridge this gap by characterizing the minimax risk of KV cache compression in terms of the intrinsic compressibility of a cache, revealing when and how accurate compression is possible. These results yield novel design principles for KV cache compression under causal masking that map efficiently to prefill and autoregressive decoding while achieving minimax-optimal risk. We instantiate these principles in a practical algorithm and report promising performance on LongBench in targeted experiments. Overall, our results provide a principled avenue for practical KV cache compression with theoretical guarantees.

KV缓存压缩理论分析推理优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。