用伪标签引导像素对比，提升无监督图像分割跨域适应能力
Pseudolabel guided pixels contrast for domain adaptive semantic segmentation
- 引入伪标签指导的像素级对比学习，增强类别内特征一致性
- 在GTA5→Cityscapes任务上提升5.1% mIoU，SYNTHIA→Cityscapes提升4.6% mIoU
- 无需增加模型复杂度，可兼容其他无监督域适应方法
语义分割对理解图像至关重要，但需大量像素级精细标注，实际获取成本高昂。无监督域适应（UDA）通过使用带标签的虚拟数据训练模型，并将其适配到无标签的真实数据。近期方法采用对比学习提升性能，但未考虑类别内部特征多样性，导致分类误差。本文分析其局限性，提出新框架Pseudo-label Guided Pixel Contrast (PGPC)，有效利用目标域图像信息，减少伪标签噪声干扰。在两个标准UDA基准上验证，基于DAFormer，在GTA5→Cityscapes和SYNTHIA→Cityscapes任务中分别实现5.1%和4.6%的相对mIoU提升。该方法还能不增加模型复杂度地增强其他UDA方法性能。代码已开源。
原文摘要 · Abstract (English)
Semantic segmentation is essential for comprehending images, but the process necessitates a substantial amount of detailed annotations at the pixel level. Acquiring such annotations can be costly in the real-world. Unsupervised domain adaptation (UDA) for semantic segmentation is a technique that uses virtual data with labels to train a model and adapts it to real data without labels. Some recent works use contrastive learning, which is a powerful method for self-supervised learning, to help with this technique. However, these works do not take into account the diversity of features within each class when using contrastive learning, which leads to errors in class prediction. We analyze the limitations of these works and propose a novel framework called Pseudo-label Guided Pixel Contrast (PGPC), which overcomes the disadvantages of previous methods. We also investigate how to use more information from target images without adding noise from pseudo-labels. We test our method on two standard UDA benchmarks and show that it outperforms existing methods. Specifically, we achieve relative improvements of 5.1% mIoU and 4.6% mIoU on the Grand Theft Auto V (GTA5) to Cityscapes and SYNTHIA to Cityscapes tasks based on DAFormer, respectively. Furthermore, our approach can enhance the performance of other UDA approaches without increasing model complexity. Code is available at https://github.com/embar111/pgpc
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。