发现量化误差可加性,提出新模型提升低精度部署效果
Saturation Makes Quantization Error Additive: A Coverage Model with a Certificate
- 基于覆盖模型构建量化损失预测框架,支持高效分配策略
- 85%-93%误差由单层效应主导,量化误差近似可加
- 适用于大模型低比特部署,尤其在4比特以下仍有效
混合精度量化需决定哪些模型部分保持高精度。现有敏感度方法(如HAWQ、CoopQ)假设层集量化损失可由孤立测量的单层或成对敏感度重构。本文在4比特权重与激活精度下测试该假设,将量化层集S的损失变化f(S)视为布尔立方上的集合函数,并通过两种经典基变换分析。结果表明:在部署分布采样的配置中,f的方差有85%-93%可由单层效应解释;对单层项之和进行单调变换后,能复现f的配置排序,最多2%配对错序。本文提出覆盖模型f(S)=c(1-∏_{i∈S}(1-a_i)),其拟合的断点率参数L即可重现测量到的方差轮廓,误差在几个百分点内。该结构支持两个预测器,每个含L+1个参数。加性模型为最优一阶预测器,其均方误差等于未被单层效应解释的方差,我们通过完整网格测量并外推至全网络规模,每项结果均附证书以证明加性模型表现上限。覆盖模型本身为第二预测器,在相同内存预算下,相较其他分配器在30B至355B参数模型上取得最低KL散度。低于4比特时,其分配方案仍可在梯度敏感度方案无法生成终止输出的预算下完成代码与推理任务。
原文摘要 · Abstract (English)
Mixed-precision quantization must decide which parts of a model to keep at higher precision. A common premise, shared by sensitivity-based methods such as HAWQ and CoopQ, is that the loss from quantizing a set of layers can be reconstructed from per-layer or pairwise sensitivities measured in isolation. We test this premise at the 4-bit weight-and-activation precisions now being deployed, treating the change in loss $f(S)$ from quantizing a layer set $S$ as a set function on the Boolean cube and analyzing it through two classical changes of basis. This analysis yields two findings. First, across configurations drawn from the deployment distribution, 85--93\% of the variance of $f$ is explained by per-layer effects alone. Second, a monotone transform of a sum of per-layer terms reproduces $f$'s ranking of configurations, misordering at most 2\% of pairs. We propose the coverage model $f(S)=c\bigl(1-\prod_{i\in S}(1-a_i)\bigr)$, which reproduces the measured variance profile of $f$ to within a few percent from its $L$ fitted break-rates. This structure supports two predictors of a configuration's loss, each with $L+1$ parameters. The additive model is the optimal first-order predictor. By Parseval's identity its mean-squared error equals the variance of $f$ left unexplained by per-layer effects, which we measure on full lattices, estimate out of sample at full-network scale, and report with every result as a certificate of how well any additive model can do. The coverage model itself is the second predictor. As allocators at matched memory, they attain the lowest KL divergence among the compared allocators on models from 30B to 355B parameters. Below four bits, the resulting allocations continue to solve code and reasoning tasks at budgets where allocations from gradient sensitivities no longer produce terminating generations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。