arXiv:2607.28589cs.CVcs.LG2026-07

根据视觉敏感度动态分配精度,让模型更省资源且不丢性能。

MixFrag: Fragility-Guided Mixed-Precision Post-Training Quantization for Vision Transformers

  • 按模块脆弱性评估动态分配比特数,避免统一精度浪费
  • 在ImageNet上保持高精度,在COCO上提升9.6点检测性能
  • 适合部署ViT到手机等低算力设备,尤其关注效率的团队

后训练量化(PTQ)已成为在资源受限设备上部署视觉变换器(ViTs)的有效方案。然而,现有方法通常对变换器各组件采用统一比特位宽,忽视其对量化的异质敏感性,导致精度分配不均。本文提出一种基于脆弱性引导的混合精度后训练量化框架——MixFrag。该方法通过小规模校准集测量全精度与孤立量化输出分布间的Kullback-Leibler(KL)散度,评估组件级量化脆弱性;随后将比特分配建模为多选背包问题(MCKP),在给定比特预算下实现自适应层间精度分配。在ImageNet-1K上对多个ViT架构的实验表明,MixFrag在实际混合精度设置下仍保持优异分类性能。此外,在COCO目标检测与实例分割任务上的评估显示,其性能达到现有混合精度PTQ方法的最先进水平,在极简配置MP3/MP3下相较最优基线提升最高达9.6 AP。额外分析验证了所提脆弱性指标的有效性及其与学习到的比特分配间的强相关性。结果表明,MixFrag是视觉变换器混合精度后训练量化的有效框架。

原文摘要 · Abstract (English)

Post-training quantization (PTQ) has emerged as an effective solution for deploying Vision Transformers (ViTs) on resource-constrained devices. However, existing PTQ methods typically employ uniform bit-widths across transformer components, overlooking their heterogeneous sensitivity to quantization and leading to inefficient precision allocation. In this paper, we propose {MixFrag, a fragility-guided mixed-precision PTQ framework for Vision Transformers. MixFrag first estimates component-level quantization fragility by measuring the Kullback--Leibler (KL) divergence between full-precision and isolated quantized output distributions using a small calibration set. It then formulates bit allocation as a Multiple-Choice Knapsack Problem (MCKP), enabling adaptive layer-wise precision assignment under a target bit budget. Extensive experiments on ImageNet-1K across multiple Vision Transformer architectures demonstrate that MixFrag achieves competitive classification performance under practical mixed-precision settings. Furthermore, evaluations on COCO object detection and instance segmentation show that MixFrag achieves state-of-the-art performance among existing mixed-precision PTQ methods, improving the previous best method by up to 9.6 AP under the challenging MP3/MP3 setting. Additional analyses validate the proposed fragility metric and demonstrate its strong correlation with the learned bit allocation. These results establish MixFrag as an effective framework for mixed-precision post-training quantization of Vision Transformers.

量化视觉变换器混合精度高效部署

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。