压缩键值缓存时，保持推理连贯性比检索更难，新方法显著提升长文本生成准确率。
Semantic Integrity Matters: Benchmarking and Preserving High-Density Reasoning in KV Cache Compression
- 分离预填充与解码阶段，将少样本示例作为整体语义单元保留
- 在长文本生成任务中准确率提升9%-18%，文档问答也表现更好
- 适用于对推理链一致性要求高的场景，如复杂逻辑推理
虽然键值（KV）缓存压缩对高效大模型推理至关重要，但现有评估过度关注稀疏检索任务，可能掩盖了高密度推理中链式思维（CoT）连贯性下降的问题。我们引入KVFundaBench系统评估这一差距，发现：尽管检索任务在高压缩下仍稳健，但推理任务在激进压缩下出现严重任务依赖性退化，因CoT链接被破坏。针对DeepSeek-R1模型的扩展分析揭示其特殊注意力模式，凸显推理链的脆弱性。基于此，我们提出ShotKV，明确分离预填充和解码阶段，优先保障语义完整性。实证表明，ShotKV在长上下文生成任务上实现9%-18%的准确率提升，并有效推广至文档问答，同时相比完整缓存推理降低11%延迟。
原文摘要 · Abstract (English)
While Key-Value (KV) cache compression is essential for efficient LLM inference, current evaluations disproportionately focus on sparse retrieval tasks, potentially masking the degradation of High-Density Reasoning where Chain-of-Thought (CoT) coherence is critical. We introduce KVFundaBench to systematically evaluate this gap, revealing a sharp dichotomy: while retrieval tasks remain robust, reasoning tasks exhibit severe Task-Dependent Degradation under aggressive compression due to disrupted CoT links. Extending our analysis to the DeepSeek-R1 model, we uncover that its specialized attention patterns offer unique insights into the fragility of reasoning chains. Guided by these findings -- specifically the necessity of preserving few-shot examples as indivisible Semantic Units -- we propose ShotKV. This approach explicitly separates prefill and decoding phases to prioritize semantic integrity. Empirical results demonstrate that ShotKV achieves 9%-18% accuracy improvements on long-context generation tasks and effectively generalizes to document QA, all while delivering an 11% latency reduction compared to full cache inference.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。