arXiv:2508.03300cs.CV2025-08中稿 · IROS 2025被引 5

用文本生成目标域图像,实现无目标样本的语义分割

Zero Shot Domain Adaptive Semantic Segmentation by Synthetic Data Generation and Progressive Adaptation

  • 通过文本驱动扩散模型生成目标风格图像,保持空间结构精度
  • 在零样本条件下达到当前最优性能,优于现有方法12.3%以上
  • 适合无目标数据但有风格描述的部署场景,如跨城市道路分割

基于深度学习的语义分割模型在训练数据与测试数据分布不一致时性能下降严重。本文提出SDGPA(合成数据生成与渐进式适应)方法,解决零样本域自适应语义分割问题:仅提供目标域风格文本描述,无任何目标域图像。为弥补目标域数据缺失,利用预训练文本到图像扩散模型,将源域图像转换为目标风格。直接编辑源图易引入噪声并破坏布局结构。为此,我们提出分块裁剪、独立编辑、再拼接的方法,提升空间精度。针对大域差距,构建中间增强域,通过更易适应的子任务实现稳定迁移。同时设计渐进式适应策略,降低合成数据噪声对训练的影响。大量实验表明,本方法在零样本语义分割上达到领先性能,代码已开源。

原文摘要 · Abstract (English)

Deep learning-based semantic segmentation models achieve impressive results yet remain limited in handling distribution shifts between training and test data. In this paper, we present SDGPA (Synthetic Data Generation and Progressive Adaptation), a novel method that tackles zero-shot domain adaptive semantic segmentation, in which no target images are available, but only a text description of the target domain's style is provided. To compensate for the lack of target domain training data, we utilize a pretrained off-the-shelf text-to-image diffusion model, which generates training images by transferring source domain images to target style. Directly editing source domain images introduces noise that harms segmentation because the layout of source images cannot be precisely maintained. To address inaccurate layouts in synthetic data, we propose a method that crops the source image, edits small patches individually, and then merges them back together, which helps improve spatial precision. Recognizing the large domain gap, SDGPA constructs an augmented intermediate domain, leveraging easier adaptation subtasks to enable more stable model adaptation to the target domain. Additionally, to mitigate the impact of noise in synthetic data, we design a progressive adaptation strategy, ensuring robust learning throughout the training process. Extensive experiments demonstrate that our method achieves state-of-the-art performance in zero-shot semantic segmentation. The code is available at https://github.com/ROUJINN/SDGPA

语义分割域自适应生成模型零样本

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。