对比CNN与大脑在情绪判断上的匹配度，发现模型仅能模仿基础视觉处理。
Assessing the Alignment of Popular CNNs to the Brain for Valence Appraisal
- 通过相关性分析比较CNN与人类行为及fMRI数据
- 模型在情绪判断上无法超越简单视觉特征提取
- 提出Object2Brain框架，揭示不同物体类别影响模型对脑对应性
卷积神经网络（CNN）在计算机视觉任务中表现优异，且被证明与人类大脑运作存在对应关系。然而，这些对应关系主要集中在一般视觉感知层面。本文首次探讨这种对应是否适用于更复杂的社交认知过程——图像情绪评估。通过相关性分析，我们评估了主流CNN架构与人类行为数据及fMRI数据在情绪判断任务中的对齐程度。结果表明，这些模型在该任务中仍局限于初级视觉处理，未能反映更高阶的大脑活动。此外，我们提出Object2Brain框架，结合GradCAM与物体检测，在滤波器层级分析不同物体类别对CNN-人脑相关性的影响。尽管整体趋势相似，不同架构显示出不同的物体类别敏感性。
原文摘要 · Abstract (English)
Convolutional Neural Networks (CNNs) are a popular type of computer model that have proven their worth in many computer vision tasks. Moreover, they form an interesting study object for the field of psychology, with shown correspondences between the workings of CNNs and the human brain. However, these correspondences have so far mostly been studied in the context of general visual perception. In contrast, this paper explores to what extent this correspondence also holds for a more complex brain process, namely social cognition. To this end, we assess the alignment between popular CNN architectures and both human behavioral and fMRI data for image valence appraisal through a correlation analysis. We show that for this task CNNs struggle to go beyond simple visual processing, and do not seem to reflect higher-order brain processing. Furthermore, we present Object2Brain, a novel framework that combines GradCAM and object detection at the CNN-filter level with the aforementioned correlation analysis to study the influence of different object classes on the CNN-to-human correlations. Despite similar correlation trends, different CNN architectures are shown to display different object class sensitivities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。