用可学习的核函数实现精准图像翻译,提升语义分割在少样本真实数据下的表现。
Image Translation with Kernel Prediction Networks for Semantic Segmentation
- 通过预测像素级变换核,构建轻量翻译函数实现语义对齐
- 多尺度判别器确保生成图像的真实感,合成数据更适配真实场景
- 适合缺乏大量真实标注的语义分割任务,尤其在合成到真实数据迁移中
语义分割依赖大量像素级标注以达到最佳性能,但由于真实数据标注困难,从业者常在大规模合成数据集上训练。无配对图像翻译可用于缓解由此产生的域差距,在低数据条件下生成更真实的训练数据。现有方法通过训练生成对抗网络(GAN)进行翻译,并利用循环一致性强制像素级语义匹配,但无法保证匹配效果,这对对噪声标签敏感的语义分割构成挑战。我们提出一种新方法——领域对抗核预测网络(DA-KPN),确保合成标签与翻译结果间的语义匹配。DA-KPN估计轻量翻译函数的像素级输入变换参数,为保证变换合理性,采用多尺度判别器区分生成样本与目标样本。实验表明,DA-KPN在合成到真实(syn2real)语义分割基准上优于以往基于GAN的方法,且在有限真实标签条件下表现更优,同时在人脸解析任务上达到相当性能。
原文摘要 · Abstract (English)
Semantic segmentation relies on many dense pixel-wise annotations to achieve the best performance, but owing to the difficulty of obtaining accurate annotations for real world data, practitioners train on large-scale synthetic datasets. Unpaired image translation is one method used to address the ensuing domain gap by generating more realistic training data in low-data regimes. Current methods for unpaired image translation train generative adversarial networks (GANs) to perform the translation and enforce pixel-level semantic matching through cycle consistency. These methods do not guarantee that the semantic matching holds, posing a problem for semantic segmentation where performance is sensitive to noisy pixel labels. We propose a novel image translation method, Domain Adversarial Kernel Prediction Network (DA-KPN), that guarantees semantic matching between the synthetic label and translation. DA-KPN estimates pixel-wise input transformation parameters of a lightweight and simple translation function. To ensure the pixel-wise transformation is realistic, DA-KPN uses multi-scale discriminators to distinguish between translated and target samples. We show DA-KPN outperforms previous GAN-based methods on syn2real benchmarks for semantic segmentation with limited access to real image labels and achieves comparable performance on face parsing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。