用零样本模型生成高质量伪标签，提升小样本语义分割效果
Zero-Shot Pseudo Labels Generation Using SAM and CLIP for Semi-Supervised Semantic Segmentation
- 通过SAM与CLIP实现零样本伪标签生成
- 在PASCAL和MS COCO数据集上显著提升分割精度
- 适合标注成本高的医学图像与自动驾驶场景
语义分割是医学图像分析和自动驾驶中的基础任务，但标注成本高昂。为此，研究者提出基于少量标注数据的半监督学习方法。该方法利用标注图像和伪标签训练分割模型，其性能取决于伪标签的质量，而伪标签质量又受目标模型性能和标注数据量影响。本文提出一种新方法：使用零样本标注技术，结合分割任意模型（SAM）与对比语言-图像预训练模型（CLIP）生成伪标签；再通过统一双流扰动方法（UniMatch）优化伪标签质量；最后以增强后的伪标签训练语义分割模型。在公开数据集PASCAL和MS COCO上的实验验证了该方法的有效性。
原文摘要 · Abstract (English)
Semantic segmentation is a fundamental task in medical image analysis and autonomous driving and has a problem with the high cost of annotating the labels required in training. To address this problem, semantic segmentation methods based on semi-supervised learning with a small number of labeled data have been proposed. For example, one approach is to train a semantic segmentation model using images with annotated labels and pseudo labels. In this approach, the accuracy of the semantic segmentation model depends on the quality of the pseudo labels, and the quality of the pseudo labels depends on the performance of the model to be trained and the amount of data with annotated labels. In this paper, we generate pseudo labels using zero-shot annotation with the Segment Anything Model (SAM) and Contrastive Language-Image Pretraining (CLIP), improve the accuracy of the pseudo labels using the Unified Dual-Stream Perturbations Approach (UniMatch), and use them as enhanced labels to train a semantic segmentation model. The effectiveness of the proposed method is demonstrated through the experiments using the public datasets: PASCAL and MS COCO. The project web page is available at: https://gsisaoki.github.io/ZERO-SHOT-PLG/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。