无需重训练,实现视觉Transformer全整数推理的量化方法。
IPTQ-ViT: Post-Training Quantization of Non-linear Functions for Integer-only Vision Transformers
- 提出多项式GELU与位移Softmax,精准逼近非线性函数。
- 在W8A8/W4A8下提升1.78%平均准确率,检测提升1.0 mAP。
- 适合部署在资源受限设备,兼容全整数推理架构。
现有视觉Transformer的量化感知训练(QAT)依赖昂贵重训以恢复非线性层量化损失,限制了其在资源受限环境的应用。现有后训练量化(PTQ)方法或仅部分量化非线性函数,或调整激活分布以维持精度,但无法实现完全整数推理。本文提出IPTQ-ViT,一种无需重训的全整数视觉Transformer PTQ框架。设计基于多项式的GELU(针对视觉数据优化)和基于位移的Softmax,以提升PTQ下的近似精度。提出统一指标,综合量化敏感度、扰动和计算成本,为每层激活选择最优近似函数。IPTQ-ViT显著优于以往PTQ方法,在图像分类上最高提升6.44%(平均1.78%)top-1准确率,目标检测提升1.0 mAP。在W8A8与W4A8条件下,超越部分浮点PTQ方法,并达到与全整数QAT相当的精度与延迟。代码将开源:https://github.com/gihwan-kim/IPTQ-ViT.git。
原文摘要 · Abstract (English)
Previous Quantization-Aware Training (QAT) methods for vision transformers rely on expensive retraining to recover accuracy loss in non-linear layer quantization, limiting their use in resource-constrained environments. In contrast, existing Post-Training Quantization (PTQ) methods either partially quantize non-linear functions or adjust activation distributions to maintain accuracy but fail to achieve fully integer-only inference. In this paper, we introduce IPTQ-ViT, a novel PTQ framework for fully integer-only vision transformers without retraining. We present approximation functions: a polynomial-based GELU optimized for vision data and a bit-shifting-based Softmax designed to improve approximation accuracy in PTQ. In addition, we propose a unified metric integrating quantization sensitivity, perturbation, and computational cost to select the optimal approximation function per activation layer. IPTQ-ViT outperforms previous PTQ methods, achieving up to 6.44\%p (avg. 1.78\%p) top-1 accuracy improvement for image classification, 1.0 mAP for object detection. IPTQ-ViT outperforms partial floating-point PTQ methods under W8A8 and W4A8, and achieves accuracy and latency comparable to integer-only QAT methods. We plan to release our code https://github.com/gihwan-kim/IPTQ-ViT.git.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。