提出非线性双极补偿,提升量化模型对异常值的鲁棒性。
Nonlinear Bipolar Compensation: Handling Outliers in Post-Training Quantization

- 在变换空间中引入非线性补偿机制,降低异常值影响。
- 使用双极对数变换压缩异常值,保持方法高效性。
- 适用于多种模型和任务,显著提升量化后准确率。
网络量化已成为最实用的模型压缩技术之一,通过将浮点数映射为低比特表示,大幅降低模型的内存与计算开销。然而,现有量化方法普遍面临速度-精度权衡问题及泛化能力有限的挑战。近年来,基于补偿的方法通过在量化网络中引入轻量级线性层,提供了一种高效且通用的解决方案,但其性能受限于补偿能力不足且对异常值敏感。本文提出非线性双极补偿(NBC),一种后训练量化方法,通过引入非线性补偿来减轻异常值的影响。我们进一步设计了双极对数变换(BLT),将量化输入与量化误差映射至变换空间以压缩异常值,并在该空间中应用简单线性层进行补偿,从而保持方法的高效性。在多种任务、模型及量化方法上的广泛实验验证了NBC方法的有效性、效率、鲁棒性与通用性。
原文摘要 · Abstract (English)
Network quantization has emerged as one of the most practical model compression techniques, which significantly reduces a model's memory and compute consumption by mapping floating-point numbers to low-bit representations. However, existing quantization methods typically suffer from the speed-accuracy tradeoff and limited generalization. To address these issues, recent compensation-based methods offer an efficient yet general solution by introducing additional lightweight linear layers into the quantized network. However, the accuracy of these methods suffers from their limited compensation capability and high sensitivity to outliers. In this paper, we propose Nonlinear Bipolar Compensation (NBC), a post-training quantization approach that introduces nonlinear compensation to reduce the effect of outliers. We further design Bipolar Logarithmic Transformation (BLT), which compresses outliers by mapping both the quantized input and the quantization error into a transformed space. A simple linear layer is then applied for compensation in the transformed space, preserving the efficiency of our method. Extensive experiments across various tasks, models, and quantization methods confirm the effectiveness, efficiency, robustness, and generality of our NBC approach.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。