提出无需训练的KV缓存压缩方法，显著降低显存占用且保持生成质量。
Squeezing the Cache, Preserving the Truth: Monotonic Equipotential Allocation with Geodesia-KV
- 基于单调块级精度分配与查询稀疏读取，实现硬件就绪的缓存压缩。
- 在PG-19上减少9.83比特/值的驻留数据，读取率降为1.95比特/值。
- 适用于大上下文生成场景，单卡16GiB显存支持100万词元输出。
现有KV缓存压缩评估混淆了驻留比特与读取带宽，并受分块教师强制伪影影响。我们提出Geodesia-KV，一种基于单调块级精度分配、精确率失真残差和查询稀疏读取的免训练缓存策略，实现真正的硬件就绪压缩。通过分离驻留与读取比特并采用因果评估，证明Geodesia-KV显著优于其他方法。在WikiText-2（16k上下文）上，其5比特操作点的困惑度低于Qwen的KIVI-4。在PG-19上，压缩版Quest模型相比基线稀疏方法，驻留比特降至9.83（原16.25），读取率降至1.95（原2.32）。作为vLLM原生插件，Geodesia-KV通过单调比特降级完全消除密集缓存驻留需求。全消费级硬件评估显示，该方法在主流架构（Qwen、Llama、DeepSeek）上可实现最高71.7%峰值显存节省，支持单卡16 GiB显存生成100万词元上下文。
原文摘要 · Abstract (English)
Current assessment of KV-cache compression performance confuses resident bits with read bandwidth and is affected by the artifacts of chunked teacher-forcing. We present Geodesia-KV, a family of training-free KV cache policies based on monotonic block-wise precision allocation, exact rate-distortion residuals, and query-sparse reading, enabling proper hardware-ready compression. With proper separation of resident and read bits and causal evaluation, we show that Geodesia-KV significantly outperforms other approaches. Specifically, on WikiText-2 with 16k context, the 5-bit operating point of Geodesia-KV results in lower perplexity at lower bitrate than KIVI-4 on Qwen. In addition, our compressed-Quest version delivers improved perplexity and reduces resident (9.83 vs 16.25 bits/value) and read rates (1.95 vs 2.32 bits/value) over baseline sparse methods on PG-19. As Geodesia-KV is implemented as native GeodesiaKVCacheManager plug-in of vLLM, Geodesia-KV fully removes the need for dense cache residency via monotonic bit demotion. With the full consumer hardware evaluation, Geodesia-KV leads to 1M-token context generation on a single 16 GiB GPU with up to 71.7% peak VRAM savings on all leading architectures (Qwen, Llama, DeepSeek).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。