arXiv:2605.13904q-bio.NCcs.LG2026-05

用特征可视化揭示大脑视觉皮层的层级组织,发现模型能生成超刺激图像。

Feature Visualization Recovers Known Cortical Selectivity from TRIBE v2

论文配图:Feature Visualization Recovers Known Cortical Selectivity from TRIBE v2
图 1 · 摘自论文原文
  • 通过梯度上升优化编码器预测激活,生成可解释的视觉刺激
  • 从V1到V4呈现空间尺度与特征复杂度递增,匹配脑区层级结构
  • 对不同脑区生成特异性模式,如面孔、运动轨迹和直线网格

脑编码模型通过预训练视觉与语言网络的内部激活来预测皮层fMRI响应,通常以保留预测准确率评估。这虽利于训练,却难以解释:仅知模型拟合数据,不知其是否内化了大脑的功能组织。本文提出特征可视化——对目标脑区(ROI)的编码器预测激活进行梯度上升——作为补充可解释性方法,并应用于TRIBE v2+V-JEPA 2(ViT-G,40层),在冻结状态和合成静态图像下对腹侧与背侧视觉通路中七个区域进行测试。在相同超参数下,该方法成功恢复了从V1至V4的空间尺度与特征复杂度递增规律,符合腹侧流层级结构;并产生三种独特下游模式:尽管仅使用静态优化,仍生成中颞区(MT)的径向“冻结运动”条纹;面部区域(FFA)生成类面孔特征;旁海马位置区(PPA)则呈现一致的直角线图案。优化后的FFA刺激使预测区域激活量约为自然人脸照片的4倍,表明生成的是对抗性超刺激而非典型样本。该方法简单、可微分,适用于任何具有可微骨干网络的脑编码模型,支持对脑编码模型的定性评估。

原文摘要 · Abstract (English)

Brain encoder models predict cortical fMRI responses from the internal activations of pretrained vision and language networks, and are typically evaluated by held-out prediction accuracy. This is a useful signal for training but a poor one for interpretation: it tells us an encoder fits the data without telling us whether it has internalized the functional organization of the brain. We propose feature visualization -- gradient ascent on the encoder's predicted activation for a target region of interest (ROI) -- as a complementary interpretability technique, and apply it to TRIBE v2 composed with V-JEPA 2 (ViT-G, 40 layers), holding both frozen and synthesizing still images for seven regions spanning the ventral and dorsal visual hierarchies. Under identical hyperparameters, the probe recovers a visible progression of increasing spatial scale and feature complexity across V1 to V4, matching the ventral-stream hierarchy. It also produces three distinctive downstream regimes: radial "frozen-motion" streaks for the middle temporal area (MT) despite static-only optimization, face-like features for the fusiform face area (FFA), and consistent rectilinear line patterns for the parahippocampal place area (PPA). Optimized FFA stimuli drive the predicted region ~4x as much as a natural face photograph, consistent with feature visualization producing adversarial super-stimuli rather than canonical exemplars. The probe is simple, differentiable, and applicable to any brain encoder with a differentiable backbone, allowing for qualitative evaluation of brain encoders.

脑科学特征可视化神经编码可解释性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。