EvolKV用进化算法动态优化大模型缓存分配，显著提升推理效率与性能。
EvolKV: Evolutionary KV Cache Compression for LLM Inference
- 通过进化搜索自动调整各层缓存预算，实现任务驱动的自适应压缩。
- 在长文本任务中优于所有基线方法，代码生成仅用1.5%缓存仍超完整缓存。
- 适合追求高效推理的大模型部署场景，尤其擅长复杂任务中的资源优化。
现有键值（KV）缓存压缩方法多依赖启发式规则，如均匀分配各层缓存或静态淘汰策略，却忽视了层间特征模式与任务性能之间的关键关联，可能导致泛化能力下降。本文提出EvolKV，一种面向层级、任务驱动的自适应缓存压缩框架，联合优化内存效率与任务表现。将缓存分配重构为多目标优化问题，利用进化搜索动态配置各层预算，并直接最大化下游任务性能。在11个任务上的大量实验表明，该方法在多种缓存预算下均优于所有基线，在长上下文任务中表现突出；在GSM8K上相比启发式基线最高提升7个百分点。尤为显著的是，其在代码补全任务中仅使用原始缓存预算的1.5%，性能已超越完整缓存设置，揭示了学习型压缩策略在缓存分配中的巨大潜力。
原文摘要 · Abstract (English)
Existing key-value (KV) cache compression methods typically rely on heuristics, such as uniform cache allocation across layers or static eviction policies, however, they ignore the critical interplays among layer-specific feature patterns and task performance, which can lead to degraded generalization. In this paper, we propose EvolKV, an adaptive framework for layer-wise, task-driven KV cache compression that jointly optimizes the memory efficiency and task performance. By reformulating cache allocation as a multi-objective optimization problem, EvolKV leverages evolutionary search to dynamically configure layer budgets while directly maximizing downstream performance. Extensive experiments on 11 tasks demonstrate that our approach outperforms all baseline methods across a wide range of KV cache budgets on long-context tasks and surpasses heuristic baselines by up to 7 percentage points on GSM8K. Notably, EvolKV achieves superior performance over the full KV cache setting on code completion while utilizing only 1.5% of the original budget, suggesting the untapped potential in learned compression strategies for KV cache budget allocation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。