通过八面体参数化优化注意力缓存压缩，实现高压缩比下零延迟解码。
OCTOPUS: Optimized KV Cache for Transformers via Octahedral Parametrization Under optimal Squared error quantization
- 将旋转后三元组方向用八面体参数化，联合量化方向与模长。
- 在所有比特率下均优于现有旋转编码器，极端压缩时优势更明显。
- 无需存储原始键值，解码时直接重建，适合部署于高吞吐场景。
键值（KV）缓存是长序列自回归推理中内存带宽和占用的主要瓶颈。近期的旋转预处理编码器（如TurboQuant、PolarQuant）表明，经过结构化随机旋转后，使用针对可解析边缘分布的逐坐标标量量化器，是近似最优的KV压缩方案。OCTOPUS在此基础上进一步推进，对旋转后的坐标三元组进行联合量化：每个三元组的方向通过八面体参数化映射至正方形，其两个坐标分量与三元组模长分别采用Lloyd-Max量化，匹配实际实现的边缘分布。优化每三元组的平方误差，获得仅依赖总键维度的非均匀比特分配。实验发现，在有限维度下该质量最优值在所有实测解码器上均为常数。该编码器为数据无关、在线、且给定种子后确定性。在文本、视频与音频任务中，OCTOPUS在所有报告比特率和指标上均达到或超越此前所有旋转编码器，尤其在极低比特率下优势愈发显著。此外，融合Triton的实现可在解码时直接重构键，无需生成未压缩键，因此不增加解码阶段的带宽或延迟。
原文摘要 · Abstract (English)
The key-value (KV) cache dominates memory bandwidth and footprint in long-context autoregressive inference. Recent rotation-preconditioned codecs (TurboQuant, PolarQuant) show that a structured random rotation followed by a per-coordinate scalar quantizer matched to an analytically tractable marginal is a near-optimal recipe for KV compression. OCTOPUS advances this paradigm through joint quantization of rotated coordinate triplets. Each triplet's direction is mapped to a square via an octahedral parameterization, and the two resulting coordinates and the triplet norm are Lloyd-Max quantized against implementation-matched marginals. Optimizing the per-triplet squared error gives a strictly non-uniform bit allocation depending only on the total dimensionality of the keys. We find the finite-dimensional quality optimum with sweeps to be constant on every real decoder we test. The codec is data-oblivious, online, and deterministic given a seed. Across text, video, and audio, OCTOPUS matches or beats every prior rotation codec at every reported bit width and metric, with a lead that grows as bits drop for extreme compression. Furthermore, a fused Triton implementation reconstructs keys on the fly without materializing the uncompressed key, so the codec adds no decode-time bandwidth or latency over the existing dequantization. Project Page: https://octopus-quant.github.io/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。