用四元数乘法压缩模型键值缓存，无需校准即可达高精度。
Hurwitz Quaternion Multiplicative Quantization for KV Cache Compression
- 将键值向量分块为四元数，通过乘法组合生成高效码本。
- 在5比特下保持接近全精度性能，部分模型比传统量化提升1900倍。
- 适合追求低存储、高推理效率的部署场景，尤其适合大模型应用。
我们提出一种无需校准的键值缓存压缩方法——赫尔维茨四元数乘法量化（HQMQ）。该方法将每个4维的键（K）或值（V）向量视为四元数，将其单位方向量化为两个四元数的乘积：一个来自24个元素的赫尔维茨群2T（即四维超立方体的24个顶点，两两夹角60°），另一个来自每层每头独立的S个随机单位四元数构成的次级码本。乘法结构实现24S个有效码字仅需存储S个参数；由于左乘是S³等距变换，初始随机码本在下游任务中性能波动小于1.5%。引入每批中位数乘子异常值提取（C=3，无校准）以应对现代模型中的异常值问题。我们在五个主流开源模型上评估：Mistral-7B（密集多头注意力）、Llama-3-8B、Qwen2.5-7B和Qwen3-8B（密集分组查询注意力）、gpt-oss-20b（稀疏MoE）。在Mistral-7B和Qwen3-8B上，HQMQ在约5比特时与fp16误差仅0.02–0.03个ppl点；在Qwen2.5-7B和Qwen3-8B上，传统int4崩溃至10⁴+ ppls，而HQMQ + Med3×恢复至接近fp16，误差0.02–0.10 ppls，仍维持在5比特左右。相比基线int4，HQMQ在所有模型上以相同比特率实现3–1900倍的性能优势；在Mistral-7B上，3.79比特即可达到与fp16相当的零样本准确率。相较于最强的有校准基线（KIVI-4，约4.5比特），在CoQA上误差仅差~1点，TruthfulQA差0.6点，GSM8K差2.3点，但比特数减少16%，且无需校准步骤。在存储层面，HQMQ可实现最高5.05倍键值缓存压缩，将Llama-3-70B（128k上下文）的缓存从43 GB压缩至8.5 GB。
原文摘要 · Abstract (English)
We propose \textbf{Hurwitz Quaternion Multiplicative Quantization (HQMQ)}, a \textbf{calibration-free} method for KV cache compression of large language models. HQMQ treats each 4-element chunk of K or V as a quaternion and quantizes its unit direction to the \emph{product} $q_p \cdot q_s$, where $q_p$ ranges over the 24-element Hurwitz group $2T$ (the 24 vertices of the 24-cell on $S^3$, pairwise angle $60^\circ$) and $q_s$ ranges over a per-(layer, head) secondary codebook of $S$ \emph{random} unit quaternions. The multiplicative composition yields $24S$ effective codewords at $S$ stored parameters; random initialization suffices because left-multiplication is an $S^3$ isometry, so seeded codebooks vary in end-task ppl by $<1.5\%$. A per-batch median-multiplier outlier extraction step ($C{=}3$, no calibration) handles modern outlier-heavy architectures. We evaluate on five modern open models: Mistral-7B (dense MHA), Llama-3-8B and Qwen2.5-7B and Qwen3-8B (dense GQA), and gpt-oss-20b (sparse MoE). On Mistral-7B and Qwen3-8B, HQMQ matches fp16 within $0.02$--$0.03$ ppl points at $\sim$5 bits. On Qwen2.5-7B and Qwen3-8B, where naive int4 collapses to $10^4{+}$ ppl, HQMQ + Med3$\times$ recovers fp16 quality within $0.02$--$0.10$ ppl points at $\sim$5 bits. HQMQ Pareto-dominates naive int by $3$--$1900\times$ at matched bits across all five models, and downstream zero-shot accuracy matches fp16 at $3.79$ bits on Mistral. Against the strongest calibrated KV-quantization baseline, HQMQ at $3.79$ bits matches KIVI-4 ($\sim 4.5$ bits) within ${\sim}1$ pt on CoQA, $0.6$ pts on TruthfulQA, and $2.3$ pts on GSM8K, at $16\%$ fewer bits and without a calibration pass. At the storage level, HQMQ delivers up to $5.05\times$ KV compression, shrinking a Llama-3-70B 128k-context cache from 43 GB to 8.5 GB.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。