提出新方法分解图像线索，评估模型对形状和纹理的依赖性及鲁棒性。
Shape Bias and Robustness Evaluation via Cue Decomposition for Image Classification and Segmentation
- 通过无AI预处理分离图像形状与纹理线索
- 在分类与分割任务中揭示模型对形状的偏好
- 首次在语义分割中评估模型对图像退化的鲁棒性
以往研究关注深度神经网络（DNN）对图像内容的感知偏差，如对纹理和形状的偏好。现有测量方法多基于风格迁移，且仅适用于图像分类任务。本文提出一种新评估流程：1）基于两种无AI数据预处理的方法，分别提取形状与纹理线索；2）设计新型基于线索分解的形状偏差评估指标，并引入对应线索分解的鲁棒性指标，用于估计DNN在图像退化下的表现。数值实验表明，分类模型的偏差结果与已有方法一致；但本文提出的鲁棒性指标在评估模型抗干扰能力上更优。此外，首次在城市景观（Cityscapes）和ADE20k语义分割数据集上揭示了分割模型的线索偏好，为理解模型行为提供新视角。
原文摘要 · Abstract (English)
Previous works studied how deep neural networks (DNNs) perceive image content in terms of their biases towards different image cues, such as texture and shape. Previous methods to measure shape and texture biases are typically style-transfer-based and limited to DNNs for image classification. In this work, we provide a new evaluation procedure consisting of 1) a cue-decomposition method that comprises two AI-free data pre-processing methods extracting shape and texture cues, respectively, and 2) a novel cue-decomposition shape bias evaluation metric that leverages the cue-decomposition data. For application purposes we introduce a corresponding cue-decomposition robustness metric that allows for the estimation of the robustness of a DNN w.r.t. image corruptions. In our numerical experiments, our findings for biases in image classification DNNs align with those of previous evaluation metrics. However, our cue-decomposition robustness metric shows superior results in terms of estimating the robustness of DNNs. Furthermore, our results for DNNs on the semantic segmentation datasets Cityscapes and ADE20k for the first time shed light into the biases of semantic segmentation DNNs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。