重新检验了CNN是否偏好纹理,发现其实更依赖局部形状。
ImageNet-trained CNNs are not biased towards texture: Revisiting feature reliance through controlled suppression
- 通过系统性抑制形状、纹理和颜色线索,量化模型依赖特征
- CNN主要依赖局部形状,现代架构可显著降低纹理依赖
- 不同领域模型依赖模式不同:视觉重形状,医疗重颜色,遥感重纹理
关于卷积神经网络(CNN)固有纹理偏好的假说,长期影响着深度学习特征使用的讨论。我们重新审视这一假说,指出Geirhos等人提出的线索冲突实验存在局限性。为此,提出一种无领域依赖的框架,通过系统性抑制形状、纹理和颜色线索来量化特征依赖,避免强制选择带来的混淆。在受控抑制条件下评估人类与神经网络,发现CNN并非固有纹理偏好,而是主要依赖局部形状特征。然而,通过现代训练策略或架构(如ConvNeXt、ViTs)可显著缓解此依赖。进一步扩展分析至计算机视觉、医学影像和遥感领域,揭示依赖模式系统性差异:计算机视觉模型优先关注形状,医学影像模型强调颜色,遥感模型则更强依赖纹理。代码已开源。
原文摘要 · Abstract (English)
The hypothesis that Convolutional Neural Networks (CNNs) are inherently texture-biased has shaped much of the discourse on feature use in deep learning. We revisit this hypothesis by examining limitations in the cue-conflict experiment by Geirhos et al. To address these limitations, we propose a domain-agnostic framework that quantifies feature reliance through systematic suppression of shape, texture, and color cues, avoiding the confounds of forced-choice conflicts. By evaluating humans and neural networks under controlled suppression conditions, we find that CNNs are not inherently texture-biased but predominantly rely on local shape features. Nonetheless, this reliance can be substantially mitigated through modern training strategies or architectures (ConvNeXt, ViTs). We further extend the analysis across computer vision, medical imaging, and remote sensing, revealing that reliance patterns differ systematically: computer vision models prioritize shape, medical imaging models emphasize color, and remote sensing models exhibit a stronger reliance on texture. Code is available at https://github.com/tomburgert/feature-reliance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。