用风格迁移降低分割模型对纹理的依赖，提升鲁棒性。
Transferring Styles for Reduced Texture Bias and Improved Robustness in Semantic Segmentation Networks
- 在图像区域上随机生成Voronoi块进行风格迁移增强
- 显著减少纹理偏差，提升对图像退化和对抗攻击的鲁棒性
- 适用于CNN与Transformer，跨城市街景与语义场景数据集
近期研究揭示了深度神经网络在图像分类中存在形状与纹理偏差，影响其泛化能力与鲁棒性。研究表明，使用风格化图像训练可降低纹理偏差并增强对图像退化的鲁棒性。本文探讨该方法在语义分割中的适用性：通过在人工图像区域（由选定数量的Voronoi细胞构成）上执行风格迁移，生成风格转移数据，用于训练分割网络，以减少对纹理线索的依赖，增强对形状特征的利用。实验表明，该方法在Cityscapes与PASCAL Context数据集上，对卷积神经网络与Transformer架构均有效，显著降低纹理偏差，并大幅提高对常见图像退化及对抗攻击的鲁棒性，验证了方法的通用性。
原文摘要 · Abstract (English)
Recent research has investigated the shape and texture biases of deep neural networks (DNNs) in image classification which influence their generalization capabilities and robustness. It has been shown that, in comparison to regular DNN training, training with stylized images reduces texture biases in image classification and improves robustness with respect to image corruptions. In an effort to advance this line of research, we examine whether style transfer can likewise deliver these two effects in semantic segmentation. To this end, we perform style transfer with style varying across artificial image areas. Those random areas are formed by a chosen number of Voronoi cells. The resulting style-transferred data is then used to train semantic segmentation DNNs with the objective of reducing their dependence on texture cues while enhancing their reliance on shape-based features. In our experiments, it turns out that in semantic segmentation, style transfer augmentation reduces texture bias and strongly increases robustness with respect to common image corruptions as well as adversarial attacks. These observations hold for convolutional neural networks and transformer architectures on the Cityscapes dataset as well as on PASCAL Context, showing the generality of the proposed method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。