基于条件数的量化方法，让图像超分辨率模型在极低比特下仍保持高精度。
CondiQuant: Condition Number Based Low-Bit Quantization for Image Super-Resolution
- 根据权重条件数设计量化误差模型，分离表征能力与敏感度
- 2~4比特下优于现有方法，参数量压缩达理论最优
- 适合需要高效部署图像超分模型的研究者和工程师
针对图像超分辨率任务的低比特模型量化长期面临精度下降问题，尤其在2~4比特时尤为显著。实验发现，精度损失主要源于激活值的量化而非权重。从数值分析角度，权重的条件数可衡量输入微小变化导致输出变化的程度，反映量化误差。为此提出CondiQuant，一种基于条件数的后训练低比特量化方法。通过将量化误差建模为权重条件数，解耦表示能力与量化敏感性，并设计高效的近端梯度下降算法，迭代最小化条件数同时保持输出不变。大量实验表明，CondiQuant在不增加计算开销的前提下，优于当前最先进的后训练量化方法，在2~4比特下实现理论最优的模型参数压缩率。代码与模型已开源。
原文摘要 · Abstract (English)
Low-bit model quantization for image super-resolution (SR) is a longstanding task that is renowned for its surprising compression and acceleration ability. However, accuracy degradation is inevitable when compressing the full-precision (FP) model to ultra-low bit widths (2~4 bits). Experimentally, we observe that the degradation of quantization is mainly attributed to the quantization of activation instead of model weights. In numerical analysis, the condition number of weights could measure how much the output value can change for a small change in the input argument, inherently reflecting the quantization error. Therefore, we propose CondiQuant, a condition number based low-bit post-training quantization for image super-resolution. Specifically, we formulate the quantization error as the condition number of weight metrics. By decoupling the representation ability and the quantization sensitivity, we design an efficient proximal gradient descent algorithm to iteratively minimize the condition number and maintain the output still. With comprehensive experiments, we demonstrate that CondiQuant outperforms existing state-of-the-art post-training quantization methods in accuracy without computation overhead and gains the theoretically optimal compression ratio in model parameters. Our code and model are released at https://github.com/Kai-Liu001/CondiQuant.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。