发现深度网络反向传播梯度与人脑视觉处理层级不匹配。
Misalignment Between Backpropagation and the Hierarchy of Brain Responses to Images

- 用fMRI和MEG分析梯度在大脑的映射模式。
- 梯度能预测高级视觉皮层的神经信号,但顺序和空间分布不符生物机制。
- 说明神经网络与大脑学习方式本质不同,适合研究认知计算的学者参考。
反向传播是深度学习的核心机制,但其是否存在于大脑中仍存争议。尽管预训练模型的前向激活可对应视觉皮层的层次结构,反向传播梯度是否也具有类似对应关系尚不清楚。本文利用功能磁共振成像(fMRI)和脑磁图(MEG)记录人类对自然图像的脑响应,将标准的前向激活编码分析扩展至反向梯度映射。以最近的自监督视觉模型DINOv3为例,并复现了八种视觉模型的结果,发现反向梯度可有效预测fMRI和MEG信号,尤其在高级视觉皮层及较晚时间点。然而,这些梯度在时空组织上偏离了生物合理反向传播机制所预期的模式:梯度计算顺序和空间分布均与人脑的时间与空间层次不一致。结果表明,尽管深度网络与大脑可能共享相似表征内容,但其学习机制可能根本不同。
原文摘要 · Abstract (English)
Backpropagation is the core learning mechanism underlying deep learning. However, whether and how this algorithm is implemented in the brain remains highly debated. In particular, while forward activations of pretrained models reliably map onto the cortical hierarchy of visual processing, it is unknown whether backpropagated gradients exhibit a similar correspondence. Here, we address this question using functional magnetic resonance imaging (fMRI) and magnetoencephalography (MEG) recordings of human brain responses to natural images. For this, we extend standard encoding analyses of forward activations to map backpropagated gradients onto neural data. Focusing on a recent self-supervised vision model (DINOv3) and reproducing results on eight vision models, we find that backpropagated gradients can reliably predict both fMRI and MEG signals, specifically in higher-level visual cortex and for later latencies. However, the spatial and temporal organization of these backpropagated gradients in the brain diverges from the patterns expected under a biologically plausible backpropagation mechanism: specifically, both the order in which gradients are computed and their spatial organization diverge from the temporal and spatial hierarchies of the human brain. Together, these results suggest that, although deep networks and the brain may share similar representational content, they likely rely on fundamentally different mechanisms to learn those representations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。