系统化优化大模型推理中的键值缓存，提升服务效率
Towards Efficient Large Language Model Serving: A Survey on System-Aware KV Cache Optimization
- 从时序、空间、结构三维度重构缓存管理机制
- 揭示缓存行为与系统目标间的协同设计潜力
- 适合关注大模型部署优化的研究者与工程师
尽管大语言模型（LLMs）发展迅速，其服务系统仍面临内存密集与成本高昂的问题。在自回归解码过程中，键值（KV）缓存用于存储键值张量，是实现低延迟、高吞吐推理服务的关键。本文聚焦于面向服务的大模型键值缓存系统（简称 sKis），从系统行为角度重新审视近期工作，将现有研究归纳为三个维度：执行与调度（时间维度）、放置与迁移（空间维度）、表示与保留（结构维度）。此外，我们分析跨行为协同设计的亲和性及行为-目标关联，揭示未来机遇。本综述系统化梳理了快速演进的领域，为理解与创新现代大模型服务架构中的缓存设计提供基础。
原文摘要 · Abstract (English)
Despite the rapid advancements of large language models (LLMs), LLM serving systems remain memory-intensive and costly. The key-value (KV) cache, which stores KV tensors during autoregressive decoding, is crucial for enabling low-latency, high-throughput LLM inference serving. In this survey, we focus on system-aware KV infrastructure for serving LLMs (abbreviated as sKis). We revisit recent work from a system behavior perspective, organizing existing efforts into three dimensions: execution and scheduling (temporal), placement and migration (spatial), and representation and retention (structural). Furthermore, we analyze cross-behavior co-design affinity and behavior-objective links, highlighting future opportunities. Our work systematizes a rapidly evolving area, providing a foundation for understanding and innovating KV cache designs in modern LLM serving infrastructure.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。