用Mamba架构构建可处理长时脑电的通用模型,兼顾效率与跨设备鲁棒性。
SAMBA: Toward a Long-Context EEG Foundation Model via Spatial Embedding and Differential Mamba
- 基于Mamba设计编码器-解码器结构,高效建模长序列脑电信号
- 在13个数据集上超越现有方法,内存和推理开销低
- 空间自适应嵌入可对齐神经生理区域,提升可解释性
长时序脑电(EEG)建模对于构建通用脑电表征模型至关重要,源于其高采样率及需捕捉长期神经活动模式的记录需求。尽管基于Transformer的模型在短序列(数秒内)表现良好,但其二次复杂度限制了在更长上下文中的扩展性。此外,不同数据集间的电极布局差异及个体间脑信号变异性,给构建通用且鲁棒的基础模型带来挑战。本文提出SAMBA,一种基于Mamba的自监督学习框架,采用U型编码器-解码器架构,有效捕捉脑电信号中的长程时间依赖与空间变异性。通过引入:(1) 时间语义随机掩码以实现语义级序列重建;(2) 多头差分Mamba模块抑制冗余并强调显著时间结构;(3) 空间自适应输入嵌入,在三维欧氏空间中学习统一嵌入,增强跨设备鲁棒性。在13个涵盖不同任务、电极配置与序列长度的脑电数据集上的实验表明,SAMBA持续优于现有先进方法,同时保持低内存消耗与推理延迟。我们还发现嵌入模块学习到的空间权重图与任务相关的神经生理区域高度一致,验证了SAMBA的可学习性与可解释性。这些结果凸显其作为实时脑机接口基础模型的可扩展性与实际潜力。
原文摘要 · Abstract (English)
Long-sequence electroencephalogram (EEG) modeling is essential for developing generalizable EEG representation models. This need arises from the high sampling rate of EEG data and the long recording durations required to capture extended neurological patterns in brain activity. Transformer-based models have shown promise in modeling short sequences of a few seconds; however, their quadratic complexity limits scalability to longer contexts. Moreover, variability in electrode montage across available datasets, along with inter-subject differences in brain signals, pose significant challenges to developing a generalizable and robust foundation model. We propose \textit{SAMBA}, a self-supervised learning framework with a Mamba-based U-shaped encoder-decoder architecture, which effectively captures long-range temporal dependencies and spatial variability in EEG data. Leveraging the inherent ability of Mamba in processing long context sizes, we introduce: (1) \textit{Temporal Semantic Random Masking} for semantic-level sequence reconstruction, (2) a \textit{Multi-Head Differential Mamba} module to suppress redundancy and emphasize salient temporal structures, and (3) a \textit{Spatial-Adaptive Input Embedding} that learns unified embeddings in a three-dimensional Euclidean space, enabling robustness across devices. Experiments on thirteen EEG datasets across diverse tasks, electrode configurations, and sequence durations demonstrate that SAMBA consistently outperforms state-of-the-art methods while maintaining low memory consumption and inference time. We also show the learned spatial weight maps from our embedding module align closely with task-relevant neurophysiological regions, demonstrating the learnability and interpretability of SAMBA. These results highlight SAMBA's scalability and practical potential as a foundation model for real-time brain-computer interface applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。