针对推理模型冗余记忆，实现高效压缩且性能不降反升
R-KV: Redundancy-aware KV Cache Compression for Reasoning Models
- 通过识别推理过程中的冗余内容，动态压缩键值缓存
- 仅用10%缓存达到近100%原性能，16%时甚至超105%
- 适合需要长推理路径的数学题或复杂任务场景
推理模型在自我反思和思维链推理方面表现优异，但常生成过长输出，导致推理时键值（KV）缓存过大。尽管思维链推理显著提升复杂任务表现，但现有缓存压缩方法在部署时可能导致推理失败。为此，我们提出专为推理模型设计的冗余感知缓存压缩方法R-KV。该方法仅需10%的原始缓存即可保留近100%的完整性能，远超现有基线（最高仅达60%）。尤为突出的是，当使用16%缓存时，性能甚至达到全缓存的105%。该压缩方案带来90%内存节省和6.6倍吞吐提升。实验表明，R-KV在两个数学推理数据集上均持续优于现有缓存压缩方法。
原文摘要 · Abstract (English)
Reasoning models have demonstrated impressive performance in self-reflection and chain-of-thought reasoning. However, they often produce excessively long outputs, leading to prohibitively large key-value (KV) caches during inference. While chain-of-thought inference significantly improves performance on complex reasoning tasks, it can also lead to reasoning failures when deployed with existing KV cache compression approaches. To address this, we propose Redundancy-aware KV Cache Compression for Reasoning models (R-KV), a novel method specifically targeting redundant tokens in reasoning models. Our method preserves nearly 100% of the full KV cache performance using only 10% of the KV cache, substantially outperforming existing KV cache baselines, which reach only 60% of the performance. Remarkably, R-KV even achieves 105% of full KV cache performance with 16% of the KV cache. This KV-cache reduction also leads to a 90% memory saving and a 6.6X throughput over standard chain-of-thought reasoning inference. Experimental results show that R-KV consistently outperforms existing KV cache compression baselines across two mathematical reasoning datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。