用智能分段压缩对话缓存，提升大模型多轮对话效率
SONIC: Segmented Optimized Nexus for Information Compression in Key-Value Caching
- 将历史对话分段压缩为语义紧凑的Nexus令牌
- 80%和50%压缩率下性能超越基线35.55%
- 支持动态内存适应，适合部署优化
键值缓存的线性增长仍是多轮大模型部署的瓶颈。现有压缩方法常忽略对话结构，依赖启发式淘汰，易丢失关键上下文。我们提出学习型框架SONIC，将历史对话段落压缩为紧凑且富含语义的Nexus令牌。通过动态预算训练，SONIC可在不重新训练的情况下灵活适配不同内存约束。实验表明，在四个多样化的多轮基准上，80%和50%压缩率下，SONIC持续优于H2O和StreamingLLM等基线。尤其在广泛使用的MTBench101基准上，平均得分较先进方法提升35.55%，验证其维持连贯多轮对话的能力。此外，SONIC显著提升部署效率，相比完整上下文生成，推理速度加快50.1%。
原文摘要 · Abstract (English)
The linear growth of Key-Value (KV) cache remains a bottleneck for multi-turn LLM deployment. Existing KV cache compression methods often fail to account for the structural properties of multi-turn dialogues, relying on heuristic eviction that risks losing critical context. We propose \textbf{SONIC}, a learning-based framework that compresses historical segments into compact and semantically rich \textbf{Nexus} tokens. By integrating dynamic budget training, SONIC allows flexible adaptation to varying memory constraints without retraining. Experiments show that at compression ratios of 80\% and 50\%, SONIC consistently outperforms baselines such as H2O and StreamingLLM on four diverse multi-turn benchmarks. Specifically, on the widely used MTBench101 benchmark, SONIC achieves an average score improvement of 35.55\% over state-of-the-art baselines, validating its effectiveness in sustaining coherent multi-turn dialogues. Furthermore, SONIC enhances deployment efficiency, accelerating the overall inference process by 50.1\% compared to full-context generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。