arXiv:2503.03088cs.CVcs.AR2025-03被引 6

提出可部署在边缘设备的SAM模型量化方法,解决四大关键难题。

AHCQ-SAM: Toward Accurate and Hardware-Compatible Post-Training Segment Anything Model Quantization

  • 设计四组件协同框架,分别应对权重病态、激活分布偏斜等问题。
  • 4比特量化下,COCO数据集上mAP提升15.2%,优于当前最优方法。
  • 支持FPGA部署,速度提升7.12倍,能效提升6.62倍,适合边缘计算。

分割一切模型(SAM)凭借强大的零样本能力革新了图像与视频分割。但其庞大的参数量和高计算需求阻碍了在资源受限边缘设备上的高效部署。后训练量化(PTQ)虽为可行方案,现有方法仍难以应对四大挑战:(1) 权重条件数差;(2) GELU后激活分布偏斜且长尾;(3) 线性投影中通道间方差显著;(4) 注意力分数指数级扩展且异质。为此,我们提出AHCQ-SAM,一个兼顾精度与硬件兼容性的PTQ框架,包含四个协同模块:(1) 激活感知条件数降低(ACNR),通过邻近点算法正则化权重矩阵以抑制病态;(2) 混合对数-均匀量化(HLUQ),结合幂次与均匀量化器捕捉偏斜激活;(3) 通道感知分组(CAG),按统计特性聚类通道,实现高精度低硬件开销;(4) 对数非线性量化(LNQ),利用对数变换自适应调整注意力分数的量化分辨率。实验表明,AHCQ-SAM在SAM上表现优异:相较于最先进方法,4比特SAM-B在COCO数据集上使用Faster R-CNN时mAP提升15.2%。此外,我们在SAM2上建立首个PTQ基准,AHCQ-SAM在4比特SAM2-Tiny上于SA-V Test数据集实现J&F提升14.01%。最后,基于FPGA的实现验证其实际价值:相比浮点基线,速度提升7.12倍,功耗效率提升6.62倍。

原文摘要 · Abstract (English)

The Segment Anything Model (SAM) has revolutionized image and video segmentation with its powerful zero-shot capabilities. However, its massive parameter scale and high computational demands hinder efficient deployment on resource-constrained edge devices. While Post-Training Quantization (PTQ) offers a practical solution, existing methods still fail to handle four critical quantization challenges: (1) ill-conditioned weights; (2) skewed and long-tailed post-GELU activations; (3) pronounced inter-channel variance in linear projections; and (4) exponentially scaled and heterogeneous attention scores. To mitigate these bottlenecks, we propose AHCQ-SAM, an accurate and hardware-compatible PTQ framework featuring four synergistic components: (1) Activation-aware Condition Number Reduction (ACNR), which regularizes weight matrices via a proximal point algorithm to suppress ill-conditioning; (2) Hybrid Log-Uniform Quantization (HLUQ), which combines power-of-two and uniform quantizers to capture skewed post-GELU activations; (3) Channel-Aware Grouping (CAG), which clusters channels with homogeneous statistics to achieve high accuracy with minimal hardware overhead; and (4) Logarithmic Nonlinear Quantization (LNQ), which utilizes logarithmic transformations to adaptively adjust quantization resolution for exponential and heterogeneous attention scores. Experimental results demonstrate that AHCQ-SAM outperforms current methods on SAM. Compared with the SOTA method, it achieves a 15.2% improvement in mAP for 4-bit SAM-B with Faster R-CNN on the COCO dataset. Furthermore, we establish a PTQ benchmark for SAM2, where AHCQ-SAM yields a 14.01% improvement in J&F for 4-bit SAM2-Tiny on the SA-V Test dataset. Finally, FPGA-based implementation validates the practical utility of AHCQ-SAM, delivering a 7.12x speedup and a 6.62x power efficiency improvement over the floating-point baseline.

量化SAM边缘计算FPGA

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。