arXiv:2409.18543cs.CV2024-09被引 1

用概率模型解决域适应中像素语义模糊问题,提升分割精度。

Reducing Semantic Ambiguity In Domain Adaptive Semantic Segmentation Via Probabilistic Prototypical Pixel Contrast

  • 将像素嵌入建模为多元高斯分布,显式表达不确定性。
  • 在昼夜转换任务中比SOTA高5.2% mIoU,增强泛化能力。
  • 无需采样重参数,高效计算分布相似性,适合实际部署。

领域自适应旨在缓解源域与目标域之间的域偏移导致的模型性能下降。尽管结合认知学习与自训练范式已取得良好效果,但在尺度、光照或重叠等情况下,确定性嵌入仍会引发语义模糊。为此,本文提出概率原型像素对比(PPPC),通过多元高斯分布对每个像素嵌入建模概率,充分挖掘其内在不确定性,从而提升模型表征质量。同时,基于后验概率估计生成原型,使决策边界远离模糊点。此外,采用高效分布相似性计算方法,避免采样与重参数化,显著降低计算开销。进一步地,动态选择图像级模糊区域,扩大对比学习中的边界点数量,有助于建立更精确的类别分布。大量实验表明,PPPC不仅有效缓解像素级模糊,实现更具区分性的表示,还在合成到真实及白天到夜间适应任务中取得显著提升,在最具挑战性的白天到夜间场景中超越此前SOTA达+5.2% mIoU,且在其他未见数据集上表现出更强泛化能力。代码与模型已公开于https://github.com/DarlingInTheSV/Probabilistic-Prototypical-Pixel-Contrast。

原文摘要 · Abstract (English)

Domain adaptation aims to reduce the model degradation on the target domain caused by the domain shift between the source and target domains. Although encouraging performance has been achieved by combining cognitive learning with the self-training paradigm, they suffer from ambiguous scenarios caused by scale, illumination, or overlapping when deploying deterministic embedding. To address these issues, we propose probabilistic proto-typical pixel contrast (PPPC), a universal adaptation framework that models each pixel embedding as a probability via multivariate Gaussian distribution to fully exploit the uncertainty within them, eventually improving the representation quality of the model. In addition, we derive prototypes from probability estimation posterior probability estimation which helps to push the decision boundary away from the ambiguity points. Moreover, we employ an efficient method to compute similarity between distributions, eliminating the need for sampling and reparameterization, thereby significantly reducing computational overhead. Further, we dynamically select the ambiguous crops at the image level to enlarge the number of boundary points involved in contrastive learning, which benefits the establishment of precise distributions for each category. Extensive experimentation demonstrates that PPPC not only helps to address ambiguity at the pixel level, yielding discriminative representations but also achieves significant improvements in both synthetic-to-real and day-to-night adaptation tasks. It surpasses the previous state-of-the-art (SOTA) by +5.2% mIoU in the most challenging daytime-to-nighttime adaptation scenario, exhibiting stronger generalization on other unseen datasets. The code and models are available at https://github.com/DarlingInTheSV/Probabilistic-Prototypical-Pixel-Contrast.

域适应语义分割概率建模对比学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。