arXiv:2607.08321cs.CV2026-07

对比CNN与ViT在纹理感知上的表现,发现ViT更接近人类视觉机制。

Texture Representations in Deep Vision Models: Comparing CNNs, Vision Transformers, and Human Perception

论文配图:Texture Representations in Deep Vision Models: Comparing CNNs, Vision Transformers, and Human Perception
图 1 · 摘自论文原文
  • 用三种纹理生成算法构建数据,通过排名统计分析模型内部表征
  • ViT对不同复杂度纹理的表征相似,且与人眼识别性能更匹配
  • 表明网络架构决定纹理表征方式,ViT比CNN更贴近人类感知

在计算视觉科学中,卷积神经网络(CNN)因其与人类及动物神经和行为数据的对齐性,被视为生物视觉的有力模型。然而,这种对齐性在超越典型语义对象识别范式、涉及复杂纹理的任务中是否依然成立仍不明确。本研究摒弃以对象为中心的视角,转而关注纹理感知这一视觉维度。我们基于同一组源图像,采用三种不同算法生成具有不同复杂度的纹理。通过排名统计方法,量化了CNN与三种视觉变换器(ViTs)内部表征所编码的信息,并比较其与人类心理物理学数据推断表征的相似性。结果表明:不同ViTs间的纹理表征高度一致;但ViTs与CNN之间的表征不一致;对于纹理识别任务,人类表现可由ViTs表征更准确预测,而非由CNN表征。综合来看,这些结果提示ViTs可能比CNN更忠实捕捉人类对纹理模式的视觉处理方式,且计算模型中的纹理表征受网络架构主导。

原文摘要 · Abstract (English)

In computational vision science, Convolutional Neural Networks (CNNs) have emerged as a popular model of biological vision because of the alignment they can exhibit with neural and behavioral data in humans and animals. However, it remains unclear to what extent this alignment persists for visual tasks that extend beyond the canonical object recognition paradigm based on well defined semantic content. In this study, we diverge from the common object-centric view by focusing on another aspect of vision: texture perception. We consider textures of different complexity generated with three different algorithms from the same source images. Using a rank-based statistic, we quantify the information encoded in the internal representations of a CNN and three Vision Transformers (ViTs), and we compare the similarity of these representations to those inferred from human psychophysics data. We find that the representation of textures is aligned in different ViTs, but not between the ViTs and the CNN; that ViTs form similar representations for textures of different complexity; that human performance in recognizing textures can be better predicted from ViTs representations rather than CNN representations. Taken together, these results suggest that ViTs may capture more faithfully than CNNs how texture patterns are visually processed by humans, and that the representations of texture stimuli in computational models may be driven by the network architecture.

视觉感知纹理识别ViTCNN

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。