对比三种缓存量化方法,发现4比特时KQV表现最佳,揭示了软最大函数放大的关键机制。
Statistical Inference and Quality Measures of KV Cache Quantisations Inspired by TurboQuant

- 基于超球面贝塔分布,分析量化对键值注意力的影响机制。
- 在4比特下KQV在所有指标上最优,5比特时QKQV几何重建更优。
- 揭示软最大函数非线性放大导致性能拐点,适合模型压缩研究者。
我们在公平的位预算下分析三种KV缓存量化方案:KV(标量MSE基线)、KQV(WHT + MSE用于K;WHT + MSE + QJL用于V)和QKQV(K、V均采用WHT + MSE + QJL)。从超球面上的贝塔分布出发,推导出对K使用QJL会使其内积方差扩大π/2倍,经软最大函数后通过Jensen不等式非线性放大。实验发现:(1) 在n=4(主流预算)时,KQV在所有分布和秩测试中均胜出,涵盖KL散度、几何K误差与6维距离;(2) K-V不对称性无条件成立:无论预算与分布如何,QKQV的KL散度始终高于KQV;(3) 存在预算依赖的交叉点:在n∈{2,3,5}时QKQV几何重建更优,而n∈{4,6}时则为KQV,与秩和尾部权重无关——构成开放的率失真问题。由构造决定仅量化K的 ext{KL}(p_{ ext{ref}} Vert p_{ ext{quant}}) 将K方向误差与路由畸变、输出坍塌关联。我们给出软最大机制超线性放大的充分条件:当n∈{2,3,5}时,该假设不成立,故QKQV占优;而在n=4时,显著更高的K误差与KL散度强烈暗示此机制是交叉点的根本原因,提供了新的解释视角。
原文摘要 · Abstract (English)
We analyse three KV cache quantization schemes under a fair bit budget: \textbf{KV} (scalar MSE baseline), \textbf{KQV} (WHT + MSE on $K$; WHT + MSE + QJL on $V$), and \textbf{QKQV} (WHT + MSE + QJL on both). Starting from the Beta distribution on the hypersphere, we trace how QJL on $K$ inflates inner product variance by $π/2$, which softmax amplifies nonlinearly via Jensen's inequality, and we present statistical inference and information metrics to highlight practical differences. Three empirical findings emerge. (1)~At $n=4$ (the practically dominant budget), KQV wins on every measure -- KL divergence, geometric $K$ error, and 6D distance -- across all distributions and ranks tested. (2)~The K--V asymmetry is unconditional: QKQV is consistently worse than KQV in KL divergence at every budget and distribution. (3)~A budget-dependent crossover exists: QKQV achieves better geometric $K$ reconstruction at $n \in \{2,3,5\}$, KQV at $n \in \{4,6\}$, invariant to rank and tail weight -- an open rate-distortion problem. $\mathrm{KL}(p_{\mathrm{ref}} \| p_{\mathrm{quant}})$, K-only by construction, bridges K direction error to routing corruption and output collapse. We present a sufficient condition when the Jensen mechanism amplifies superlinearly through the softmax. At $n \in \{2,3,5\}$, QKQV wins geometrically because this assumption does not bind. At $n=4$, elevated K error and KL divergence for QKQV strongly suggest the Jensen mechanism is the operative cause of the crossover, providing a new perspective and explanation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。