用极坐标变换实现高效量化，让大模型长文本记忆节省4倍以上。
PolarQuant: Quantizing KV Caches with Polar Transformation
- 将KV缓存转为极坐标，用角度值做量化，避免传统方法的归一化步骤。
- 在长文本任务中压缩缓存超4.2倍，性能优于现有最优方法。
- 适合需要节省显存的长序列生成场景，如超长文档处理。
大型语言模型（LLMs）在处理长上下文时需存储大量键值（KV）嵌入，占用显著内存。量化是减少内存消耗的常用手段。本文提出PolarQuant，一种结合随机预处理与极坐标变换的新量化方法。该方法通过高效递归算法将KV嵌入转换为极坐标，并对得到的角度进行量化。关键洞察在于：经随机预处理后，极坐标下的角度分布紧密且高度集中，具有可解析计算的形式。这一优良分布无需显式归一化，从而避免了传统量化方法中需以全精度存储量化参数（如零点、缩放因子）所带来的额外内存开销。PolarQuant跳过归一化步骤，实现显著内存节省。长上下文评估显示，PolarQuant使KV缓存压缩超过4.2倍，同时在质量指标上优于当前最先进的方法。
原文摘要 · Abstract (English)
Large language models (LLMs) require significant memory to store Key-Value (KV) embeddings in their KV cache, especially when handling long-range contexts. Quantization of these KV embeddings is a common technique to reduce memory consumption. This work introduces PolarQuant, a novel quantization method employing random preconditioning and polar transformation. Our method transforms the KV embeddings into polar coordinates using an efficient recursive algorithm and then quantizes resulting angles. Our key insight is that, after random preconditioning, the angles in the polar representation exhibit a tightly bounded and highly concentrated distribution with an analytically computable form. This nice distribution eliminates the need for explicit normalization, a step required by traditional quantization methods which introduces significant memory overhead because quantization parameters (e.g., zero point and scale) must be stored in full precision per each data block. PolarQuant bypasses this normalization step, enabling substantial memory savings. The long-context evaluation demonstrates that PolarQuant compresses the KV cache by over x4.2 while achieving the best quality scores compared to the state-of-the-art methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。