提出GSRQ量化方法，让大模型缓存压缩到1比特以下仍保持高精度。
GSRQ: Gain-Shape Residual Quantization for Sub-1-bit KV Cache
- 用改进的聚类算法提升方向保留能力，解决传统方法的向量偏移问题。
- 在LLaMA-3-8B上1比特压缩时，长文本任务准确率从11.34提升至33.54。
- 适合关注大模型推理加速与内存优化的研究者和工程师。
大语言模型在长上下文场景下面临键值（KV）缓存内存线性增长的瓶颈。向量量化（VQ），尤其是残差量化（RQ），是将KV缓存压缩至1比特以下的可行方案，通过小码本逐级编码残差。然而，现有方法普遍依赖标准ℓ₂ K-means作为核心码本学习机制。我们发现该机制存在高维隐藏问题：欧氏中心平均会导致中心点收缩，削弱ℓ₂失真中的角度对齐项，影响方向保真度。为此，我们提出增益-形状K均值（GSKM），可直接替代标准K-means，提升方向保真度，同时保持甚至优于原ℓ₂失真性能。在此基础上，构建了增益-形状残差量化（GSRQ），将加权版GSKM嵌入RQ流程。在LLaMA-3-8B上，GSRQ在多个比特率下显著优于现有基线。在1比特时，LongBench任务平均准确率从11.34提升至33.54，较VQLLM提升22.20个百分点。
原文摘要 · Abstract (English)
The deployment of Large Language Models (LLMs) with extended context windows is increasingly constrained by the linear growth of Key-Value (KV) cache memory. Vector Quantization (VQ), particularly Residual Quantization (RQ), is a promising approach for pushing KV cache storage toward the sub-1-bit regime by progressively encoding residuals with small codebooks. However, most VQ methods still rely on standard $\ell_2$ $K$-means as the core codebook-learning primitive. We identify a subtle high-dimensional issue of this primitive: Euclidean centroid averaging can induce centroid shrinkage, which weakens the angular alignment term in the $\ell_2$ distortion and makes directional preservation harder. To address this issue, we propose Gain-Shape $K$-means (GSKM), a drop-in replacement for $K$-means that improves directional fidelity while matching, and in some regimes improving, $\ell_2$ distortion. We then build Gain-Shape Residual Quantization (GSRQ) by incorporating a weighted extension of GSKM into an RQ pipeline. On LLaMA-3-8B, GSRQ substantially improves over strong KV cache quantization baselines across bit rates. At 1-bit, it improves the average accuracy across LongBench tasks from 11.34 to 33.54, a gain of 22.20 percentage points over VQLLM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。