arXiv:2410.05341eess.IVcs.AI2024-10NeurIPS被引 22

用脑电图生成全脑功能磁共振图像,实现跨模态精准映射。

NeuroBOLT: Resting-state EEG-to-fMRI Synthesis with Multi-dimensional Feature Mapping

  • 通过时序、空间、频域多维度特征学习,实现脑电到脑磁的跨模态转换。
  • 在感知、认知及深部脑区均实现高精度重建,优于现有方法。
  • 适用于不同任务与站点,为神经科学研究提供新工具。

功能性磁共振成像(fMRI)是现代神经科学中不可或缺的工具,可在毫米级空间分辨率下非侵入式观测全脑动态。然而,其高昂成本和固定设备限制了应用。随着跨模态合成与脑解码技术的发展,深度神经网络为直接从更易获取、便携的脑电图(EEG)推断全脑高分辨率fMRI信号提供了新路径。然而,神经活动到fMRI血流动力学响应的复杂映射以及EEG的空间模糊性,给建模与可解释性带来挑战。目前研究多局限于少数脑区或单一状态(如静息态或特定任务),难以推广至其他脑区或条件。为此,本文提出新型通用框架NeuroBOLT(Neuro-to-BOLD Transformer),利用时序、空间、频域多维表征学习,将原始EEG数据映射为全脑fMRI信号。实验表明,NeuroBOLT能有效重建未见过的静息态下初级感觉、高级认知及深部皮层脑区的fMRI信号,达到当前最优性能,并具备跨条件与跨站点的泛化潜力,显著推动两模态融合发展。

原文摘要 · Abstract (English)

Functional magnetic resonance imaging (fMRI) is an indispensable tool in modern neuroscience, providing a non-invasive window into whole-brain dynamics at millimeter-scale spatial resolution. However, fMRI is constrained by issues such as high operation costs and immobility. With the rapid advancements in cross-modality synthesis and brain decoding, the use of deep neural networks has emerged as a promising solution for inferring whole-brain, high-resolution fMRI features directly from electroencephalography (EEG), a more widely accessible and portable neuroimaging modality. Nonetheless, the complex projection from neural activity to fMRI hemodynamic responses and the spatial ambiguity of EEG pose substantial challenges both in modeling and interpretability. Relatively few studies to date have developed approaches for EEG-fMRI translation, and although they have made significant strides, the inference of fMRI signals in a given study has been limited to a small set of brain areas and to a single condition (i.e., either resting-state or a specific task). The capability to predict fMRI signals in other brain areas, as well as to generalize across conditions, remain critical gaps in the field. To tackle these challenges, we introduce a novel and generalizable framework: NeuroBOLT, i.e., Neuro-to-BOLD Transformer, which leverages multi-dimensional representation learning from temporal, spatial, and spectral domains to translate raw EEG data to the corresponding fMRI activity signals across the brain. Our experiments demonstrate that NeuroBOLT effectively reconstructs unseen resting-state fMRI signals from primary sensory, high-level cognitive areas, and deep subcortical brain regions, achieving state-of-the-art accuracy with the potential to generalize across varying conditions and sites, which significantly advances the integration of these two modalities.

脑电图功能磁共振跨模态生成模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。