用低秩残差补偿提升低精度矩阵乘法精度，误差降1-2个量级
A method of using RSVD in residual calculation of LowBit GEMM
- 在低精度矩阵乘法中引入低秩残差补偿机制
- 4比特下图像分类准确率提升至61.8%，误差降1-2个量级
- 无需额外数据，可无缝集成到现有低精度计算流程
近年来硬件技术的进步为低精度应用带来了诸多可能。然而，低精度运算会引入显著的计算误差，对保持计算精度构成重大挑战。本文提出低秩残差量化矩阵乘法（LRQMM）方法，在密集低精度量化矩阵乘法中引入低秩近似进行残差补偿。该方法仅需BLAS-2级别额外开销，即可实现数倍精度提升。此外，LRQMM是一种完全无数据的量化方法，无需额外训练数据。其仅作用于低精度GEMM算子，易于与其他方法集成。实验表明，对于大矩阵规模，LRQMM可将直接量化矩阵乘法的误差降低1~2个数量级，计算速度仅下降约20%。在深度学习网络中，LRQMM-4bit在ResNet-50上达到61.8%的ImageNet Top-1准确率，而直接量化仅为8.3%。
原文摘要 · Abstract (English)
The advancements of hardware technology in recent years has brought many possibilities for low-precision applications. However, the use of low precision can introduce significant computational errors, posing a considerable challenge to maintaining the computational accuracy. We propose low-rank residuals quantized matrix multiplication(LRQMM) method which introduces low-rank approximation in residual compensation for dense low precision quantization matrix multiplication. It can bring several times accuracy improvement with only BLAS-2 level extra time overhead. Moreover, LRQMM is a completely data-free quantization method that does not require additional data for pre-training. And it only works with low precision GEMM operator, which is easy to couple with other methods. Through experimentation, LRQMM can reduce the error of direct quantized matrix multiplication by 1~2 orders of magnitude, when dealing with larger matrix sizes, the computational speed is only reduced by approximately 20\%. In deep learning networks, LRQMM-4bit achieves 61.8% ImageNet Top-1 accuracy in Resnet-50, while the Direct Quant accuracy is only 8.3%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。