用角度量化压缩注意力缓存，实现近无损效果。
TurboAngle: Near-Lossless KV Cache Compression via Uniform Angle Quantization
- 在快速沃尔什-哈达玛域中对角度进行量化，结合随机旋转使元素分布均匀。
- 7个模型中6个达到近无损质量，每元素仅需3.28至3.67角度比特。
- 支持按层配置精度，识别关键层与负迁移层，适合大模型推理优化。
我们通过在快速沃尔什-哈达玛域中量化角度来压缩KV缓存，其中随机对角旋转使连续元素对近似均匀分布在单位圆上。我们扩展了该角度量化器，引入逐层早提升机制，可独立配置每层的K和V码本大小，将更高精度分配给模型特定的关键层。在七个模型（1B至7B参数）中，逐层早提升在四个模型上实现无损压缩，在七中六个达到近无损质量，每元素仅需3.28至3.67角度比特。非对称归一化量化（键为8比特，值为4比特对数空间）在Mistral-7B上实现每元素6.56比特，困惑度增加+0.0014，且无需校准数据。层组敏感性分析揭示了模型特异性的瓶颈模式，包括以键为主或以值为主的层，以及精度提升反而降低质量的负迁移层。
原文摘要 · Abstract (English)
We compress KV cache entries by quantizing angles in the Fast Walsh-Hadamard domain, where a random diagonal rotation makes consecutive element pairs approximately uniformly distributed on the unit circle. We extend this angular quantizer with per-layer early-boost, which independently configures K and V codebook sizes at each layer, allocating higher precision to a model-specific subset of critical layers. Across seven models (1B to 7B parameters), per-layer early-boost achieves lossless compression on four models and near-lossless quality on six of seven, at 3.28 to 3.67 angle bits per element. Asymmetric norm quantization (8-bit for keys, 4-bit log-space for values) yields 6.56 total bits per element on Mistral-7B with perplexity degradation of +0.0014 and no calibration data. A layer-group sensitivity analysis reveals model-specific bottleneck patterns, including K-dominated versus V-dominated layers and negative-transfer layers where increased precision degrades quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。