用反事实刺激验证大脑如何真正表征视觉概念
From Activation to Specificity: Automating Counterfactual Testing of Visual Representations in the Human Brain

- 通过生成可控图像并测试大脑对概念的特异性响应
- 发现仅激活强不代表真实表征,30%以上定位是假阳性
- 适合神经科学与脑-机接口研究者使用
确定人脑中哪些区域表征特定视觉概念是神经科学的核心挑战。现有方法通过激活最大化定位粗粒度功能区(如人脸、场景),但强激活未必表示该区域真正表征该概念,可能仅由相关视觉或语义线索驱动。我们提出BrainTRACE框架,结合生成模型与脑成像模型,自动构建包含目标概念图像、移除目标概念的反事实编辑图像及候选混淆因素图像的刺激集。利用图像到fMRI编码模型预测脑响应,搜索对目标概念具有特异性响应的区域。该方法成功复现已知功能定位,并在数十个概念上识别新候选区域,验证于预测和实测fMRI数据。关键发现:若无反事实检验,大量定位为假阳性,证明仅靠激活不足以证明表征存在。
原文摘要 · Abstract (English)
Identifying which brain regions represent a visual concept in the human brain is a central challenge in neuroscience. Existing approaches have localized coarse functional regions (e.g., faces, places) through activation maximization, identifying regions that activate strongly for a target concept relative to other concepts. Yet strong activation alone does not establish that a region represents the concept itself, as responses may instead be driven by correlated visual or semantic cues. We introduce BrainTRACE (Testing Representations through Counterfactual Evidence), an automated framework that combines generative and brain models to synthesize controlled stimuli and validate neural representations through targeted counterfactual-specificity testing. Given a query specifying a concept of interest, our framework constructs targeted stimulus sets comprising concept images, counterfactual edits that remove the target concept while preserving other image content, and images with candidate correlated distractors. It then uses an image-to-fMRI encoding model to predict brain responses and searches for representations that respond specifically to the target concept over correlated alternatives. BrainTRACE returns validated candidate representations and proposes follow-up fMRI experiments to further test or extend its discoveries. Our approach successfully recovers known functional localizations and identifies new candidate representations across dozens of concepts, validated on both predicted and measured fMRI data. Critically, we show that without counterfactual evaluation, a large fraction of localizations would be false positives, confirming that activation alone is insufficient evidence of representation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。