arXiv:2507.11955cs.CV2025-07中稿 · IEEE Transactions …被引 4

提出新方法提升语义分割在未知场景的泛化能力。

Prototypical Progressive Alignment and Reweighting for Generalizable Semantic Segmentation

  • 分阶段对齐特征,从易到难逐步缩小域间差距
  • 通过原型重加权机制降低无关数据的负面影响
  • 基于CLIP构建双原型,增强模型对未知域的适应性

通用语义分割旨在对未见目标域保持良好性能,是真实应用中高泛化性的关键挑战。类别原型作为类别中心,因其稳定性和语义一致性,可作为域不变线索促进泛化。但现有方法存在三方面问题:一是粗粒度原型对齐策略影响性能;二是源域特征平均生成的朴素原型易过拟合,受无关数据干扰;三是忽略不同样本适应难度差异,一视同仁处理。为此,本文提出新型框架PPAR(原型渐进对齐与重加权),利用CLIP模型强泛化能力。定义原始文本原型(OTP)与视觉文本原型(VTP),通过CLIP生成作为对齐基础。设计渐进对齐策略,按难易程度逐级对齐特征,逐步减少域间差距。提出原型重加权机制,评估源数据可靠性并动态调整其贡献,缓解无关或有害特征带来的负迁移。还提供了理论分析,证明方法与域泛化理论的一致性。多基准测试表明,PPAR达到当前最优性能,验证有效性。

原文摘要 · Abstract (English)

Generalizable semantic segmentation aims to perform well on unseen target domains, a critical challenge due to real-world applications requiring high generalizability. Class-wise prototypes, representing class centroids, serve as domain-invariant cues that benefit generalization due to their stability and semantic consistency. However, this approach faces three challenges. First, existing methods often adopt coarse prototypical alignment strategies, which may hinder performance. Second, naive prototypes computed by averaging source batch features are prone to overfitting and may be negatively affected by unrelated source data. Third, most methods treat all source samples equally, ignoring the fact that different features have varying adaptation difficulties. To address these limitations, we propose a novel framework for generalizable semantic segmentation: Prototypical Progressive Alignment and Reweighting (PPAR), leveraging the strong generalization ability of the CLIP model. Specifically, we define two prototypes: the Original Text Prototype (OTP) and Visual Text Prototype (VTP), generated via CLIP to serve as a solid base for alignment. We then introduce a progressive alignment strategy that aligns features in an easy-to-difficult manner, reducing domain gaps gradually. Furthermore, we propose a prototypical reweighting mechanism that estimates the reliability of source data and adjusts its contribution, mitigating the effect of irrelevant or harmful features (i.e., reducing negative transfer). We also provide a theoretical analysis showing the alignment between our method and domain generalization theory. Extensive experiments across multiple benchmarks demonstrate that PPAR achieves state-of-the-art performance, validating its effectiveness.

语义分割域泛化CLIP原型学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。