动态调整视觉变压器量化精度,提升边缘设备部署效果。
ScalePredictor: Instance-aware Scale Learning for Accurate Quantization of Vision Transformers

- 根据浅层激活分布预测深层量化尺度,实现动态量化。
- 在ImageNet上比现有方法准确率更高,且计算开销极小。
- 适合需要高效压缩ViT模型的边缘计算场景。
视觉变压器在多个领域表现卓越,但其高计算需求使其在边缘设备上的部署仍具挑战。后训练量化(PTQ)通过少量校准数据实现模型压缩,几乎无额外训练成本。然而,现有多数PTQ方法采用对所有样本统一的静态量化策略,而自然图像间激活分布差异显著,导致该方法本质次优。本文提出ScalePredictor,一种面向视觉变压器的动态量化框架,用于精确高效的量化尺度学习。我们首次揭示浅层激活分布范围与深层最优量化尺度间的隐含关联,据此设计一种高效范围提取机制,在浅层捕捉鲁棒的范围统计,并通过泰勒启发的多项式投影模块,一次性生成全部量化尺度。基于多项式逼近的高效性,ScalePredictor引入可忽略的计算开销,避免耗时的即时校准。在ImageNet上的大量实验表明,ScalePredictor持续优于先前的PTQ方法,实现了更优的准确率-效率权衡。代码与附加结果见补充材料。
原文摘要 · Abstract (English)
Vision Transformers have achieved remarkable success in many fields, yet their deployment on edge devices remains challenging due to their substantial computational demands. Post-Training Quantization (PTQ) offers an attractive solution by compressing models using a small calibration set with minimal training overhead. However, most existing PTQ works adopt a static quantization paradigm that is uniformly applied to all instances. Given the substantial diversity of natural images, the activation distributions vary significantly across samples, making these methods inherently suboptimal. In this paper, we propose ScalePredictor, a dynamic quantization framework for accurate and efficient quantization scale learning of ViTs. We first reveal a hidden correlation between the distribution range of shallow-layer activations and the optimal scales of deeper layers. Based on this, we develop a scale learning mechanism that integrates an efficient range extraction approach to capture robust range statistics at the shallow stage, which are then fed into a Taylor-motivated polynomial scale projection module to generate all quantization scales simultaneously. With the efficiency of polynomial approximation, ScalePredictor introduces insignificant computational overhead while avoiding costly just-in-time calibration. Extensive experiments on ImageNet demonstrate that ScalePredictor consistently outperforms prior PTQ methods, achieving a more favorable accuracy-efficiency trade-off. Code and additional results are shown in the supplementary materials.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。