arXiv:2501.16410cs.CV2025-01ICLR被引 3

无需标注,动态对齐跨域语义标签,实现更细粒度分割。

DynAlign: Unsupervised Dynamic Taxonomy Alignment for Cross-Domain Segmentation

  • 利用基础模型先验知识动态对齐源域与目标域的标签
  • 在GTA→Mapillary Vistas和GTA→IDD上显著提升分割精度
  • 支持新类别、细粒度类别或命名差异的自动适应

当前无监督域自适应(UDA)方法通常假设源域与目标域具有相同的类别标签,忽略了真实场景中常见的标签级域差异,限制了其对更细粒度或新类别识别的能力。为此,本文提出DynAlign框架,将UDA与基础模型结合,同时弥合图像级和标签级的域差距。该方法利用基础模型的语义先验知识,将源域类别动态对齐至目标域中可能为新类别、更细粒度或命名不同的类别(如 vehicle 到 {car, truck, bus}),并基于此进行精确分割与类别重分配。进一步提出一种动态知识融合策略,根据场景上下文自适应调整对齐结果。实验在道路场景分割基准GTA→Mapillary Vistas和GTA→IDD上验证了方法有效性,显著优于现有方法。代码将公开。

原文摘要 · Abstract (English)

Current unsupervised domain adaptation (UDA) methods for semantic segmentation typically assume identical class labels between the source and target domains. This assumption ignores the label-level domain gap, which is common in real-world scenarios, thus limiting their ability to identify finer-grained or novel categories without requiring extensive manual annotation. A promising direction to address this limitation lies in recent advancements in foundation models, which exhibit strong generalization abilities due to their rich prior knowledge. However, these models often struggle with domain-specific nuances and underrepresented fine-grained categories. To address these challenges, we introduce DynAlign, a framework that integrates UDA with foundation models to bridge both the image-level and label-level domain gaps. Our approach leverages prior semantic knowledge to align source categories with target categories that can be novel, more fine-grained, or named differently (e.g., vehicle to {car, truck, bus}). Foundation models are then employed for precise segmentation and category reassignment. To further enhance accuracy, we propose a knowledge fusion approach that dynamically adapts to varying scene contexts. DynAlign generates accurate predictions in a new target label space without requiring any manual annotations, allowing seamless adaptation to new taxonomies through either model retraining or direct inference. Experiments on the street scene semantic segmentation benchmarks GTA to Mapillary Vistas and GTA to IDD validate the effectiveness of our approach, achieving a significant improvement over existing methods. Our code will be publicly available.

无监督学习语义分割跨域对齐基础模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。