用信息瓶颈理论重设计键值缓存淘汰，提升长文本生成质量
Rethinking KV Cache Eviction via a Unified Information-Theoretic Objective
- 基于信息瓶颈原理，推导出可量化缓存信息容量的闭式目标函数
- 在多个模型和长文本任务中，相比旧方法提升生成准确率10%以上
- 适合需要高效处理长上下文的大模型推理场景
键值（KV）缓存对大语言模型推理至关重要，但其内存开销成为长上下文生成的关键瓶颈。现有淘汰策略多依赖经验启发式，缺乏严格的理论基础。本文从信息瓶颈视角重新思考KV缓存淘汰，在线性高斯注意力近似下推导出闭式互信息目标，刻画了保留缓存子集的有效信息容量。该公式表明，多种现有策略可被解释为同一容量最大化原则的不同近似。据此提出CapKV，一种基于统计杠杆率的对数行列式近似方法，直接以信息保真为目标进行缓存淘汰。实验在多个模型和长文本基准上验证，CapKV consistently优于先前方法，在内存效率与生成保真度间取得更优平衡。代码已开源。
原文摘要 · Abstract (English)
Key-Value (KV) caching is essential for large language model inference, yet its memory overhead poses a critical bottleneck for long-context generation. Existing eviction policies predominantly rely on empirical heuristics, lacking a rigorous theoretical foundation. This work rethinks KV cache eviction through the lens of the Information Bottleneck principle. Under a linear-Gaussian surrogate of attention, we derive a closed-form mutual information objective that characterizes the effective information capacity of a retained KV cache subset. This formulation reveals that a wide range of existing eviction strategies can be interpreted as different approximations of the same capacity-maximization principle. Guided by this insight, we introduce CapKV, a capacity-aware eviction method that directly targets information preservation via a log-determinant approximation using statistical leverage scores. This approach replaces heuristic selection with a theoretically grounded mechanism that preserves the maximum predictive signal. Extensive experiments across multiple models and long-context benchmarks show that CapKV consistently outperforms prior methods, achieving a better trade-off between memory efficiency and generational fidelity. Our code is available at https://github.com/jiamingyy/CapKV
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。