为视觉语言分割设计高效量化框架，让小设备也能跑得快准
PTQ4RIS: Post-Training Quantization for Referring Image Segmentation
- 针对视觉与文本编码器分别设计双区域量化和重排序保留异常值方法
- 4~8位量化下在3个数据集上性能领先，最低损失仅1.2% mAP
- 首个专为指代图像分割设计的后训练量化方案，适合边缘部署
指代图像分割（RIS）旨在通过理解视觉与语言信息，在图像中定位被句子提及的物体。然而，现有RIS方法多聚焦于高性能模型，忽视了资源受限的边缘设备上的实际应用需求，导致设备端推理面临挑战。为此，我们提出一种高效且有效的后训练量化框架PTQ4RIS。首先深入分析了RIS模型量化性能下降的根本原因，并针对视觉与文本编码器分别提出双区域量化（DRQ）与基于重排序的异常值保留量化（RORQ），以应对量化难题。在三个不同基准数据集上，采用4至8比特设置进行的大量实验表明其性能显著优于现有方法。尤为重要的是，PTQ4RIS是首个专为RIS任务设计的后训练量化方法，验证了该技术在RIS应用中的可行性。代码与演示视频见：https://github.com/gugu511yy/PTQ4RIS。
原文摘要 · Abstract (English)
Referring Image Segmentation (RIS), aims to segment the object referred by a given sentence in an image by understanding both visual and linguistic information. However, existing RIS methods tend to explore top-performance models, disregarding considerations for practical applications on resources-limited edge devices. This oversight poses a significant challenge for on-device RIS inference. To this end, we propose an effective and efficient post-training quantization framework termed PTQ4RIS. Specifically, we first conduct an in-depth analysis of the root causes of performance degradation in RIS model quantization and propose dual-region quantization (DRQ) and reorder-based outlier-retained quantization (RORQ) to address the quantization difficulties in visual and text encoders. Extensive experiments on three benchmarks with different bits settings (from 8 to 4 bits) demonstrates its superior performance. Importantly, we are the first PTQ method specifically designed for the RIS task, highlighting the feasibility of PTQ in RIS applications. Code and video are available at {https://github.com/gugu511yy/PTQ4RIS}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。