首个统一脑电与脑磁信号的通用基础模型,支持跨设备、跨模态分析。
BrainOmni: A Brain Foundation Model for Unified EEG and MEG Signals
- 首创脑信号分词器,将时空脑活动转为离散表示,兼容不同传感器布局。
- 基于1997小时脑电+656小时脑磁数据预训练,实现跨模态性能超越现有模型。
- 适合神经科学、医疗诊断及跨设备脑机接口研究者使用。
脑电图(EEG)和脑磁图(MEG)通过捕捉树突电流产生的电磁场,非侵入性地测量神经活动。尽管源于相同生物物理原理,两者信号模式差异显著,且受传感器配置和设备差异影响。现有方法多依赖特定模态与数据集的独立模型,限制了性能与跨域扩展能力。本文提出BrainOmni,首个可泛化于异构EEG与MEG记录的脑基础模型。为统一多样化数据源,我们引入BrainTokenizer——首个将时空脑活动量化为离散表示的分词器。其核心是新型传感器编码器,能编码空间布局、方向与类型等传感器属性,实现跨设备与模态兼容。基于离散表示,BrainOmni通过自监督预训练学习统一的脑信号语义嵌入。据我们所知,它是首个同时支持EEG与MEG信号的基础模型,也是首个进行大规模MEG预训练的模型。预训练数据来自公开资源,涵盖1,997小时的EEG和656小时的MEG。实验表明,BrainOmni在多种下游任务中优于现有基础模型与最先进的专用模型,并展现出对未见设备的强大泛化能力。进一步分析显示,联合EEG-MEG(EMEG)训练在两种模态上均带来持续提升。代码与检查点已公开于https://github.com/OpenTSLab/BrainOmni。
原文摘要 · Abstract (English)
Electroencephalography (EEG) and magnetoencephalography (MEG) measure neural activity non-invasively by capturing electromagnetic fields generated by dendritic currents. Although rooted in the same biophysics, EEG and MEG exhibit distinct signal patterns, further complicated by variations in sensor configurations across modalities and recording devices. Existing approaches typically rely on separate, modality- and dataset-specific models, which limits the performance and cross-domain scalability. This paper proposes BrainOmni, the first brain foundation model that generalises across heterogeneous EEG and MEG recordings. To unify diverse data sources, we introduce BrainTokenizer,the first tokenizer that quantises spatiotemporal brain activity into discrete representations. Central to BrainTokenizer is a novel Sensor Encoder that encodes sensor properties such as spatial layout, orientation, and type, enabling compatibility across devices and modalities. Building upon the discrete representations, BrainOmni learns unified semantic embeddings of brain signals by self-supervised pretraining. To the best of our knowledge, it is the first foundation model to support both EEG and MEG signals, as well as the first to incorporate large-scale MEG pretraining. A total of 1,997 hours of EEG and 656 hours of MEG data are curated and standardised from publicly available sources for pretraining. Experiments show that BrainOmni outperforms both existing foundation models and state-of-the-art task-specific models on a range of downstream tasks. It also demonstrates strong generalisation to unseen EEG and MEG devices. Further analysis reveals that joint EEG-MEG (EMEG) training yields consistent improvements across both modalities. Code and checkpoints are publicly available at https://github.com/OpenTSLab/BrainOmni.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。