跨模态无监督域适应新方法，实现不同模态间知识迁移。
Heterogeneous-Modal Unsupervised Domain Adaptation via Latent Space Bridging
- 设计双分支结构，通过特征一致性和域对齐损失实现跨模态对齐
- 在六个基准数据集上达到当前最优性能，验证方法有效性
- 适合需要在异构模态间迁移语义分割知识的研究者
无监督域适应（UDA）方法能有效缓解域间差异，但在源域与目标域属于完全不同的模态时表现受限。为此，我们提出一种新设定——异构模态无监督域适应（HMUDA），通过引入包含两种模态未标注样本的桥接域，实现跨模态知识迁移。针对该设定，我们提出潜空间桥接（LSB）框架，专用于语义分割任务。LSB采用双分支结构，结合特征一致性损失以对齐不同模态的表示，并利用域对齐损失减小各分类中心间的域差异。在六个基准数据集上的大量实验表明，LSB实现了当前最佳性能。
原文摘要 · Abstract (English)
Unsupervised domain adaptation (UDA) methods effectively bridge domain gaps but become struggled when the source and target domains belong to entirely distinct modalities. To address this limitation, we propose a novel setting called Heterogeneous-Modal Unsupervised Domain Adaptation (HMUDA), which enables knowledge transfer between completely different modalities by leveraging a bridge domain containing unlabeled samples from both modalities. To learn under the HMUDA setting, we propose Latent Space Bridging (LSB), a specialized framework designed for the semantic segmentation task. Specifically, LSB utilizes a dual-branch architecture, incorporating a feature consistency loss to align representations across modalities and a domain alignment loss to reduce discrepancies between class centroids across domains. Extensive experiments conducted on six benchmark datasets demonstrate that LSB achieves state-of-the-art performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。