用脑皮层曲面建模提升视觉刺激重建精度
From Flat to Round: Redefining Brain Decoding with Surface-Based fMRI and Cortex Structure
- 将fMRI信号视为皮层曲面上的2D球面数据，保留空间一致性
- 融合结构MRI实现个体化解码，准确率显著提升
- 适合神经科学与医学影像领域研究者参考
从人脑活动（如fMRI）中重构视觉刺激，连接了神经科学与计算机视觉。现有方法常忽略脑结构-功能关系，将空间信息平面化并忽视个体解剖差异。为此，我们提出：(1) 新型球面分词器，将fMRI信号显式建模为皮层表面的2D球面数据；(2) 融合结构MRI（sMRI）数据，实现个体解剖变异的个性化编码；(3) 正样本混合策略，高效利用同一视觉刺激对应多个fMRI扫描。这些创新共同提升了重建准确性、生物学可解释性及跨个体泛化能力。实验表明，性能优于当前最优方法，验证了该生物启发方法的有效性与可解释性。
原文摘要 · Abstract (English)
Reconstructing visual stimuli from human brain activity (e.g., fMRI) bridges neuroscience and computer vision by decoding neural representations. However, existing methods often overlook critical brain structure-function relationships, flattening spatial information and neglecting individual anatomical variations. To address these issues, we propose (1) a novel sphere tokenizer that explicitly models fMRI signals as spatially coherent 2D spherical data on the cortical surface; (2) integration of structural MRI (sMRI) data, enabling personalized encoding of individual anatomical variations; and (3) a positive-sample mixup strategy for efficiently leveraging multiple fMRI scans associated with the same visual stimulus. Collectively, these innovations enhance reconstruction accuracy, biological interpretability, and generalizability across individuals. Experiments demonstrate superior reconstruction performance compared to SOTA methods, highlighting the effectiveness and interpretability of our biologically informed approach.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。