FlowKV通过隔离缓存避免重复压缩，显著提升长对话一致性
FlowKV: Enhancing Multi-Turn Conversational Coherence in LLMs via Isolated Key-Value Cache Management
- 将历史对话缓存分段隔离，仅压缩最新一轮内容
- 在10轮以上对话中指令遵循准确率从10.9%提升至75.4%
- 无需训练，兼容任意缓存压缩方法，适合长对话系统
大型语言模型在多轮对话应用中日益普及，但键值（KV）缓存管理成为关键瓶颈。随着对话历史增长，KV缓存线性膨胀，带来巨大计算开销；现有淘汰策略常因反复压缩早期上下文导致信息丢失和记忆遗忘。本文提出FlowKV，一种无需训练的多轮隔离机制，可适配任意KV缓存压缩方法。其核心在于保留过往轮次已压缩的缓存，仅对最新完成轮次新生成的键值对进行压缩，有效避免旧上下文被重复压缩，缓解灾难性遗忘。实验表明，FlowKV在保持指令遵循准确率和用户偏好保留方面显著优于基线，在10轮以上对话中准确率从10.90%提升至75.40%，尤其在后期对话中表现突出。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are increasingly deployed in multi-turn conversational applications, where the management of the Key-Value (KV) Cache presents a significant bottleneck. The linear growth of the KV Cache with dialogue history imposes substantial computational costs, and existing eviction strategies often degrade performance by repeatedly compressing early conversational context, leading to information loss and context forgetting. This paper introduces FlowKV, a novel \textbf{multi-turn isolation mechanism} for KV Cache management, which can be applied to any KV Cache compression method without training. FlowKV's core innovation is a multi-turn isolation mechanism that preserves the accumulated compressed KV cache from past turns. Compression is then strategically applied only to the newly generated KV pairs of the latest completed turn, effectively preventing the re-compression of older context and thereby mitigating catastrophic forgetting. Our results demonstrate that FlowKV consistently and significantly outperforms baseline strategies in maintaining instruction-following accuracy and user preference retention from 10.90\% to 75.40\%, particularly in later conversational turns.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。