按层重要性与敏感度分配比特数，实现更高效的视觉Transformer量化。
Mix-QViT: Mixed-Precision Vision Transformer Quantization Driven by Layer Importance and Quantization Sensitivity
- 基于层重要性和量化敏感度动态分配每层比特数
- 3-6比特下性能超越现有方法，2比特混合精度表现最优
- 适合需要高效部署的视觉Transformer模型开发者
本文提出Mix-QViT，一种可解释性驱动的混合精度量化框架，通过两个标准为各层分配比特数：一是使用层间相关性传播（LRP）评估层重要性，识别每层对最终分类的贡献；二是通过在不同精度下量化单层并保持其他层为基准，评估量化敏感度。针对训练后量化（PTQ），引入裁剪通道量化方法，通过去除层归一化后激活的极端异常值，降低通道间剧烈差异的影响。我们在ViT、DeiT和Swin Transformer模型上验证该方法，在多个数据集上测试。实验表明，无论是固定比特还是混合比特方案，均在3位、4位和6位精度下优于现有技术；在量化感知训练中，2位混合精度也达到更优性能。
原文摘要 · Abstract (English)
In this paper, we propose Mix-QViT, an explainability-driven MPQ framework that systematically allocates bit-widths to each layer based on two criteria: layer importance, assessed via Layer-wise Relevance Propagation (LRP), which identifies how much each layer contributes to the final classification, and quantization sensitivity, determined by evaluating the performance impact of quantizing each layer at various precision levels while keeping others layers at a baseline. Additionally, for post-training quantization (PTQ), we introduce a clipped channel-wise quantization method designed to reduce the effects of extreme outliers in post-LayerNorm activations by removing severe inter-channel variations. We validate our approach by applying Mix-QViT to ViT, DeiT, and Swin Transformer models across multiple datasets. Our experimental results for PTQ demonstrate that both fixed-bit and mixed-bit methods outperform existing techniques, particularly at 3-bit, 4-bit, and 6-bit precision. Furthermore, in quantization-aware training, Mix-QViT achieves superior performance with 2-bit mixed-precision.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。