用4000个通用原子稀疏编码，大幅压缩大模型缓存内存。
Lexico: Extreme KV Cache Compression via Sparse Coding over Universal Dictionaries
- 用通用词典+稀疏编码，直接控制稀疏度实现灵活压缩。
- 在多个模型上仅用15-25%内存，保持90-95%原始性能。
- 低内存下仍有效，比2比特量化压缩提升1.7倍。
我们提出Lexico，一种基于通用字典的稀疏编码KV缓存压缩方法。核心发现是现代大模型的键值缓存可由约4000个与输入无关的原子进行稀疏线性组合准确近似，从而实现跨不同提示、任务和模型的高效压缩。采用正交匹配追踪进行稀疏逼近，通过直接控制稀疏度实现灵活压缩比率。在GSM8K数据集上，对Mistral、Llama 3、Qwen2.5等多个模型家族，Lexico仅使用15-25%的完整缓存内存，便维持90-95%的原始性能，优于量化和令牌驱逐方法。尤其在低内存场景下，2比特量化失效时，Lexico在LongBench和GSM8K上仍实现最高达1.7倍的压缩提升，同时保持高精度。
原文摘要 · Abstract (English)
We introduce Lexico, a novel KV cache compression method that leverages sparse coding with a universal dictionary. Our key finding is that key-value cache in modern LLMs can be accurately approximated using sparse linear combination from a small, input-agnostic dictionary of ~4k atoms, enabling efficient compression across different input prompts, tasks and models. Using orthogonal matching pursuit for sparse approximation, Lexico achieves flexible compression ratios through direct sparsity control. On GSM8K, across multiple model families (Mistral, Llama 3, Qwen2.5), Lexico maintains 90-95% of the original performance while using only 15-25% of the full KV-cache memory, outperforming both quantization and token eviction methods. Notably, Lexico remains effective in low memory regimes where 2-bit quantization fails, achieving up to 1.7x better compression on LongBench and GSM8K while maintaining high accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。