首个全整数运算的ViT语义分割框架,轻量高效易部署。
I-Segmenter: Integer-Only Vision Transformer for Efficient Semantic Segmentation
- 全用整数运算替代浮点计算,模型更轻更快。
- 在低精度下仍保持95%以上基线准确率,推理提速1.2倍。
- 无需复杂校准,单图即可实现良好性能,适合边缘设备。
视觉变换器(ViTs)在语义分割任务中表现优异,但其高内存和计算开销限制了在资源受限设备上的部署。量化是提升效率的有效手段,但基于ViT的分割模型对低精度极为敏感,量化误差会在深层编码器-解码器结构中累积。本文提出I-Segmenter,首个完全基于整数运算的ViT分割框架。在Segmenter基础上,系统性地将浮点运算替换为整数运算。为稳定训练与推理,提出λ-ShiftGELU激活函数,缓解均匀量化对长尾激活分布的处理缺陷。同时移除L2归一化层,并将解码器中的双线性插值替换为最近邻上采样,确保整个计算图仅含整数操作。大量实验表明,I-Segmenter平均比FP32基线低5.1%准确率,模型大小减少最高3.8倍,优化后推理速度提升1.2倍。即使在仅需单张校准图的一次性量化(PTQ)下,也能达到竞争力的性能,凸显其实际部署价值。
原文摘要 · Abstract (English)
Vision Transformers (ViTs) have recently achieved strong results in semantic segmentation, yet their deployment on resource-constrained devices remains limited due to their high memory footprint and computational cost. Quantization offers an effective strategy to improve efficiency, but ViT-based segmentation models are notoriously fragile under low precision, as quantization errors accumulate across deep encoder-decoder pipelines. We introduce I-Segmenter, the first fully integer-only ViT segmentation framework. Building on the Segmenter architecture, I-Segmenter systematically replaces floating-point operations with integer-only counterparts. To further stabilize both training and inference, we propose $λ$-ShiftGELU, a novel activation function that mitigates the limitations of uniform quantization in handling long-tailed activation distributions. In addition, we remove the L2 normalization layer and replace bilinear interpolation in the decoder with nearest neighbor upsampling, ensuring integer-only execution throughout the computational graph. Extensive experiments show that I-Segmenter achieves accuracy within a reasonable margin of its FP32 baseline (5.1 % on average), while reducing model size by up to 3.8x and enabling up to 1.2x faster inference with optimized runtimes. Notably, even in one-shot PTQ with a single calibration image, I-Segmenter delivers competitive accuracy, underscoring its practicality for real-world deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。