arXiv:2502.05034cs.CV2025-02ICML被引 19

用显式对齐提升跨人脑解码精度,尤其适合数据少的情况

MindAligner: Explicit Brain Functional Alignment for Cross-Subject Visual Decoding from Limited fMRI Data

  • 通过脑信号转移矩阵实现跨被试信号对齐
  • 在有限fMRI数据下显著提升视觉重建准确率
  • 适合脑科学与跨人脑建模研究者

脑解码旨在从fMRI信号中重建人类视觉感知,对理解大脑感知机制至关重要。现有方法受限于单被试范式,因脑间差异大导致跨被试泛化能力弱,且训练成本高,加之fMRI数据稀缺。为此,我们提出MindAligner,一种基于显式功能对齐的跨被试脑解码框架,适用于有限fMRI数据。首先,学习一个脑转移矩阵(BTM),将任意新被试的脑信号映射到已知被试空间,实现预训练解码模型的无缝复用;其次,设计脑功能对齐模块,在不同视觉刺激下通过多层级对齐损失进行软对齐,揭示高可解释性的细粒度功能对应关系。实验表明,该方法在数据受限条件下显著优于现有方法,并提供有价值的跨被试功能分析洞见。代码将公开。

原文摘要 · Abstract (English)

Brain decoding aims to reconstruct visual perception of human subject from fMRI signals, which is crucial for understanding brain's perception mechanisms. Existing methods are confined to the single-subject paradigm due to substantial brain variability, which leads to weak generalization across individuals and incurs high training costs, exacerbated by limited availability of fMRI data. To address these challenges, we propose MindAligner, an explicit functional alignment framework for cross-subject brain decoding from limited fMRI data. The proposed MindAligner enjoys several merits. First, we learn a Brain Transfer Matrix (BTM) that projects the brain signals of an arbitrary new subject to one of the known subjects, enabling seamless use of pre-trained decoding models. Second, to facilitate reliable BTM learning, a Brain Functional Alignment module is proposed to perform soft cross-subject brain alignment under different visual stimuli with a multi-level brain alignment loss, uncovering fine-grained functional correspondences with high interpretability. Experiments indicate that MindAligner not only outperforms existing methods in visual decoding under data-limited conditions, but also provides valuable neuroscience insights in cross-subject functional analysis. The code will be made publicly available.

脑解码fMRI跨被试对齐

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。