通过预处理优化量化矩阵乘法误差,显著提升低精度推理性能。
Contraction-Gauge Preconditioning for Quantized Matrix Multiplication
- 提出收缩-规范预处理框架,联合优化因子表示与共享模式。
- 8位下误差降低18.0%,4位下降低20.5%,对齐真实误差排名相关性达0.937。
- 提供可验证的最优选择方法,适合部署低精度AI模型的研发者。
我们研究了双因子量化下的低精度矩阵乘法C=AB。推导出在独立零均值逐元素误差(已知方差场)下的期望平方乘积误差的精确有限维恒等式;该式对非溢出减法抖动和独立随机舍入成立,并实证评估确定性四舍五入(RTN)。利用乘积保持等价关系AB=(AT)(T^{-1}B),提出收缩-规范预处理:在量化前联合选择因子表示及其共享模式。预处理可降低乘积误差,但可能需额外存储反向操作数的变换后量化副本——共享变换需1份,块级变换最多每块1份。在正对角规范(折叠)族内,几何规划求解全局最优共享折叠,线性规划判断单位折叠是否已达最优。对其他族,推导可计算的选择统计量——缩放用尾指数、分块用轮廓展宽、旋转用相干性与加权格拉姆能量、层级深度用切片能量协方差,并给出候选排序的上界。在训练好的三块图像分类器的十二个线性乘积中,抖动模型预测与确定性RTN误差的组内秩相关性在8位为0.937,4位为0.918。几何规划折叠在8位下使保留误差降低18.0%(几何均值),4位下降低20.5%;优于SmoothQuant风格网格基线,在十二个乘积中十项胜出;并使组合逻辑损失均方误差降低15.4%和26.4%。因此,本文提供精确的随机乘积误差计账、对角族内的可证明选择,以及基于RTN的可重用变换候选统一评估目标。
原文摘要 · Abstract (English)
We study low-precision computation of C=AB with both factors quantized. We derive an exact finite-dimensional identity for the expected squared product error under independent, zero-mean entrywise errors with known variance fields; it holds exactly for non-overloading subtractive dither and for independent stochastic rounding, and we empirically assess deterministic round-to-nearest (RTN). Using the product-preserving equivalence AB=(AT)(T^{-1}B), we formulate contraction-gauge preconditioning: jointly choosing a factor representation and its sharing pattern before quantization. Preconditioning can reduce product error but may require extra transformed, quantized copies of the opposite operand: a shared transform needs one copy, a block-specific transform up to one per block. Within the bounded family of positive diagonal gauges (folds), a geometric program computes a globally optimal shared fold and a linear program decides whether the identity fold is already optimal. For other families we derive computable selection statistics -- tail index for scaling, profile spread for partitioning, coherence and weighted-Gram energy for rotations, slice-energy covariance for hierarchy depth -- with upper bounds for ranking heuristic candidates. Across twelve linear products from a trained three-block image classifier, median within-product rank correlations between dither-model predictions and deterministic-RTN errors are 0.937 at 8 bits and 0.918 at 4 bits. The GP fold cuts held-out product error over the identity fold by 18.0% (8-bit) and 20.5% (4-bit) in geometric mean, beats a SmoothQuant-style grid baseline at both precisions and on ten of twelve products, and lowers composed logit MSE by 15.4% and 26.4%. We thus provide exact stochastic product-error accounting, certified selection within the diagonal family, and a common objective for evaluating reusable transform candidates under RTN.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。