通过对比与上下文感知学习，提升无监督域适应语义分割性能
C^2DA: Contrastive and Context-aware Domain Adaptive Semantic Segmentation
- 引入跨域对比损失，增强图像内像素关联性
- 改进混合策略并结合掩码图像建模，利用类别间上下文关系
- 在GTA-V→Cityscapes等任务上显著超越现有方法
无监督域自适应语义分割（UDA-SS）旨在仅用源域数据（如合成数据）训练模型，并将其适配到目标域（如真实世界）而无需目标标注。现有方法多关注域间知识迁移，却忽略了图像内在结构及两域像素分布特性，也常忽视上下文信息。为此，本文提出一种同时学习域内与上下文感知知识的UDA-SS框架。通过在两个域中引入对比损失，使同类别像素聚拢、异类像素分离，强化图像内部像素关联；通过融合类别间上下文依赖改进混合策略，并采用掩码图像建模（MIM）技术，利用有限掩码信息增强视觉识别鲁棒性。大量实验表明，本方法在GTA-V→Cityscapes和Synthia→Cityscapes任务上分别提升0.51%和0.54% mIoU，优于当前最优方法。代码已开源。
原文摘要 · Abstract (English)
Unsupervised domain adaptive semantic segmentation (UDA-SS) aims to train a model on the source domain data (e.g., synthetic) and adapt the model to predict target domain data (e.g., real-world) without accessing target annotation data. Most existing UDA-SS methods only focus on inter-domain knowledge to mitigate the data-shift problem. However, learning the inherent structure of the images and exploring the intrinsic pixel distribution of both domains are ignored, which prevents the UDA-SS methods from producing satisfactory performance like supervised learning. Moreover, incorporating contextual knowledge is also often overlooked. Considering these issues, in this work, we propose a UDA-SS framework that learns both intra-domain and context-aware knowledge. To learn the intra-domain knowledge, we incorporate contrastive loss in both domains, which pulls pixels of similar classes together and pushes the rest away, facilitating intra-image-pixel-wise correlations. To learn context-aware knowledge, we modify the mixing technique by leveraging contextual dependency among the classes. Moreover, we adapt the Mask Image Modeling (MIM) technique to properly use context clues for robust visual recognition, using limited information about the masked images. Comprehensive experiments validate that our proposed method improves the state-of-the-art UDA-SS methods by a margin of 0.51% mIoU and 0.54% mIoU in the adaptation of GTA-V->Cityscapes and Synthia->Cityscapes, respectively. We open-source our C2DA code. Code link: github.com/Masrur02/C-Squared-DA
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。