训练平坦的全精度模型可显著降低量化误差，提升低比特推理性能。
Efficiently Training A Flat Neural Network Before It has been Quantizated
- 通过分离激活与权重量化误差，将其建模为独立高斯噪声。
- 采用噪声注入优化方法，使模型达到平坦最小值，减少量化偏差。
- 适用于追求高效低比特推理的视觉Transformer部署场景。
针对视觉变换器（ViTs）的后训练量化（PTQ）因模型压缩效率高而备受关注。然而，现有方法通常忽略良好训练的神经网络与量化模型之间的关系，导致量化误差较大。目前尚不清楚如何高效训练一种与具体低比特模型无关的神经网络。本文首次发现，平坦的全精度神经网络对低比特量化至关重要。为此，我们提出一个框架，通过测量并解耦误差来源，主动预处理模型。具体而言，将激活量化误差（AQE）和权重量化误差（WQE）分别建模为独立的高斯噪声，并研究多种噪声注入优化方法以实现平坦最小值。实验验证了该方法的有效性，为获取低比特PTQ模型开辟了新路径。
原文摘要 · Abstract (English)
Post-training quantization (PTQ) for vision transformers (ViTs) has garnered significant attention due to its efficiency in compressing models. However, existing methods typically overlook the relationship between a well-trained NN and the quantized model, leading to considerable quantization error for PTQ. However, it is unclear how to efficiently train a model-agnostic neural network which is tailored for a predefined precision low-bit model. In this paper, we firstly discover that a flat full precision neural network is crucial for low-bit quantization. To achieve this, we propose a framework that proactively pre-conditions the model by measuring and disentangling the error sources. Specifically, both the Activation Quantization Error (AQE) and the Weight Quantization Error (WQE) are statistically modeled as independent Gaussian noises. We study several noise injection optimization methods to obtain a flat minimum. Experimental results attest to the effectiveness of our approach. These results open novel pathways for obtaining low-bit PTQ models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。