提出一种在潜在空间重建语义特征的新方法,提升无监督域适应分割性能。
Masked Representation Modeling for Domain-Adaptive Segmentation
- 在潜空间直接掩码并重建高层语义特征,目标与分割任务对齐。
- 在GTA→Cityscapes和Cityscapes→Synthia上分别提升2.3和2.8 mIoU。
- 轻量级重建模块零推理开销,适配DeepLab、DAFormer等主流模型。
无监督域适应(UDA)语义分割旨在将带标签的源域模型迁移到无标签的目标域。尽管对比学习等自监督任务提升了特征判别性,但掩码建模因架构限制和目标错位仍被忽视。本文提出掩码表示建模(MRM),在潜空间直接进行表示掩码与重建,不依赖低层信号(如像素或视觉标记)。该方法聚焦高层语义特征,目标与分割一致,并可无缝集成至DeepLab和DAFormer等标准架构。为支持高效重建,设计轻量辅助模块Rebuilder,与分割网络联合训练但推理时移除,实现零测试开销。大量实验表明,MRM在多种架构与UDA基准上持续提升性能。集成四个代表性基线后,在GTA→Cityscapes上平均提升+2.3 mIoU,Cityscapes→Synthia上提升+2.8 mIoU,证明其为简单、有效且通用的无监督域适应分割策略。
原文摘要 · Abstract (English)
Unsupervised domain adaptation (UDA) for semantic segmentation seeks to transfer models from a labeled source domain to an unlabeled target domain. While auxiliary self-supervised tasks such as contrastive learning have enhanced feature discriminability, masked modeling remains underexplored due to architectural constraints and misaligned objectives. We propose Masked Representation Modeling (MRM), an auxiliary task that performs representation masking and reconstruction directly in the latent space. Unlike prior masked modeling methods that reconstruct low-level signals (e.g., pixels or visual tokens), MRM targets high-level semantic features, aligning its objective with segmentation and integrating seamlessly into standard architectures like DeepLab and DAFormer. To support efficient reconstruction, we design a lightweight auxiliary module, Rebuilder, which is jointly trained with the segmentation network but removed during inference, introducing zero test-time overhead. Extensive experiments demonstrate that MRM consistently improves segmentation performance across diverse architectures and UDA benchmarks. When integrated with four representative baselines, MRM achieves an average gain of +2.3 mIoU on GTA $\rightarrow$ Cityscapes and +2.8 mIoU on Cityscapes $\rightarrow$ Synthia, establishing it as a simple, effective, and generalizable strategy for unsupervised domain-adaptive semantic segmentation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。