预调优提升低比特量化模型精度,无需复杂训练
Efficient Tuning Before Low-Bit Post-Training Quantization for Stochastic Gradient Descent-optimized Models

- 在量化前对模型施加量化误差扰动进行优化,使其更适应后续压缩
- 2比特权重+4比特激活下,小图像数据集精度提升2.14%,城市景观分割提升5.80%
- 适用于各类基于随机梯度下降的模型,兼容任意量化后端
后训练量化(PTQ)可在有限内存和算力条件下压缩深度神经网络。然而,低比特(如2比特或4比特)PTQ常导致性能显著下降。现有方法多在未约束的全精度(FP)模型上操作,主要通过事后重构缓解量化误差。本文指出,低比特PTQ性能受限不仅源于量化误差最小化,更与全精度模型对量化误差的容忍度有关。为此提出高效量化前调优(ETBQ),针对随机梯度下降优化的模型,在PTQ前加入预调理阶段。该阶段在权重与激活量化误差分布中采样扰动,引导模型优化至对后续量化不敏感的损失曲面区域。不同于量化感知训练(QAT),ETBQ不需训练伪量化部署模型,计算与内存开销低,输出的全精度模型可适配任意PTQ后端。在CIFAR-100、Tiny-ImageNet、ImageNet和Cityscapes上的实验表明,ETBQ在多种任务中一致提升低比特量化性能。例如在W2A4设置下,于Tiny-ImageNet上较朴素PTQ提升2.14%准确率,于Cityscapes上提升5.80% mIoU。代码已开源。
原文摘要 · Abstract (English)
Post-training quantization (PTQ) compresses deep neural networks for deployment under limited memory and computational budgets. However, low-bit (i.e., 2-bit or 4-bit) PTQ often suffers from substantial performance degradation. Most existing PTQ methods operate on an unconstrained full-precision (FP) model and primarily address quantization errors through post-hoc reconstruction. We argue that low-bit PTQ accuracy is limited not only by post-quantization error minimization, but also by the quantization-error tolerance of a FP model itself. In this paper, we propose Efficient Tuning Before Quantization (ETBQ), a pre-conditioning tuning stage for Stochastic Gradient Descent (SGD)-optimized models before PTQ. During tuning, the FP model is optimized under perturbations sampled from the error distributions of weight and activation quantization, guiding the model toward a loss-landscape region that is less sensitive to the subsequent PTQ. Unlike QAT, ETBQ does not train a fake-quantized deployment model, which is computationally and memory intensive. Instead, ETBQ outputs a FP model that can be used by any PTQ backend. Experiments on CIFAR-100, Tiny-ImageNet, ImageNet, and Cityscapes provide consistent evidence that ETBQ improves low-bit PTQ across diverse tasks. Under W2A4 settings, e.g., ETBQ improves over naive PTQ by 2.14\% top-1 accuracy on Tiny-ImageNet and by 5.80\% mIoU on Cityscapes. Code is available at https://github.com/xpxpxp2001xpxpxp/ETBQ.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。