arXiv:2412.12050cs.CV2024-12AAAI被引 20

提升跨域分割的语义一致性与风格多样性,解决现有方法误判和模糊问题。

Exploring Semantic Consistency and Style Diversity for Domain Generalized Semantic Segmentation

  • 通过语义查询增强和文本驱动风格变换,提升跨域语义一致性
  • 在未见数据集上达49.11 mIoU,比之前最优方法高4.08 mIoU
  • 适合需要强泛化能力的自动驾驶、遥感图像分割场景

领域泛化语义分割(DGSS)旨在仅使用源域数据提升对未知目标域的分割泛化能力。现有方法主要依赖特征归一化和领域随机化,但存在显著局限:特征归一化易混淆语义特征,导致分类错误;领域随机化常引入无关风格噪声,造成分割模糊。为此,本文提出新框架SCSD,包含三个关键组件:首先,设计语义查询增强模块,提升掩码解码器中对象查询的语义感知与区分能力,实现跨域语义一致性预测;其次,提出文本驱动风格变换模块,利用领域差异文本嵌入可控引导图像特征风格转换,增强域间风格多样性;最后,引入风格协同优化机制,通过协同加权风格对比损失与风格聚合损失,强化域间特征分离与域内特征聚合,防止相似域特征空间坍缩。大量实验表明,所提SCSD显著优于现有最先进方法。以GTAV为源域训练,在四个未见域数据集上平均达49.11 mIoU,超越前序最优方法4.08 mIoU。代码已公开于https://github.com/nhw649/SCSD。

原文摘要 · Abstract (English)

Domain Generalized Semantic Segmentation (DGSS) seeks to utilize source domain data exclusively to enhance the generalization of semantic segmentation across unknown target domains. Prevailing studies predominantly concentrate on feature normalization and domain randomization, these approaches exhibit significant limitations. Feature normalization-based methods tend to confuse semantic features in the process of constraining the feature space distribution, resulting in classification misjudgment. Domain randomization-based methods frequently incorporate domain-irrelevant noise due to the uncontrollability of style transformations, resulting in segmentation ambiguity. To address these challenges, we introduce a novel framework, named SCSD for Semantic Consistency prediction and Style Diversity generalization. It comprises three pivotal components: Firstly, a Semantic Query Booster is designed to enhance the semantic awareness and discrimination capabilities of object queries in the mask decoder, enabling cross-domain semantic consistency prediction. Secondly, we develop a Text-Driven Style Transform module that utilizes domain difference text embeddings to controllably guide the style transformation of image features, thereby increasing inter-domain style diversity. Lastly, to prevent the collapse of similar domain feature spaces, we introduce a Style Synergy Optimization mechanism that fortifies the separation of inter-domain features and the aggregation of intra-domain features by synergistically weighting style contrastive loss and style aggregation loss. Extensive experiments demonstrate that the proposed SCSD significantly outperforms existing state-of-theart methods. Notably, SCSD trained on GTAV achieved an average of 49.11 mIoU on the four unseen domain datasets, surpassing the previous state-of-the-art method by +4.08 mIoU. Code is available at https://github.com/nhw649/SCSD.

语义分割域泛化风格多样性跨域一致

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。