用线性复杂度模型实现跨电极布局的高效脑电建模。
LuMamba: Latent Unified Mamba for Electrode Topology-Invariant and Efficient EEG Modeling
- 采用潜空间统一编码与双向Mamba块,解决电极布局差异问题。
- 仅460万参数达80.99%平衡准确率,377倍降低计算量。
- 适合脑电异常检测、阿尔茨海默病识别等临床场景使用。
脑电图(EEG)在临床和神经技术中实现无创脑活动监测,但构建基础模型面临电极拓扑差异与计算可扩展性挑战,传统Transformer存在序列复杂度二次增长问题。为此,我们提出LuMamba(潜空间统一Mamba),一种结合拓扑不变编码与线性复杂度状态空间建模的自监督框架,利用LUNA的查询学习交叉注意力机制实现通道统一,采用FEMBA的双向Mamba块进行高效时序建模。该架构首次系统研究了潜空间-欧氏联合嵌入预测架构(LeJEPA)在生物信号学习中的应用。在超过21,000小时未标注的TUEG语料库上预训练,评估涵盖16至26通道配置下的异常检测、伪影识别与精神状态分类等五项下游任务。预训练中,仅掩码重建生成结构化但泛化性差的表示,而仅LeJEPA产生弥散嵌入;两者结合取得最稳健性能。仅460万参数下,于TUAB数据集达80.99%平衡准确率,阿尔茨海默病检测0.97 AUPR,计算量仅为现有模型的1/377,且可处理12倍更长序列而不超典型显存限制。代码已开源。
原文摘要 · Abstract (English)
Electroencephalography (EEG) enables non-invasive monitoring of brain activity across clinical and neurotechnology applications, yet building foundation models for EEG remains challenging due to differing electrode topologies and computational scalability, as Transformer architectures incur quadratic sequence complexity. As a joint solution, we propose LuMamba (Latent Unified Mamba), a self-supervised framework combining topology-invariant encodings with linear-complexity state-space modeling, using LUNA's learned-query cross-attention mechanism for channel unification, and FEMBA's bidirectional Mamba blocks for efficient temporal modeling. Within this architecture, we provide the first systematic investigation of the Latent-Euclidean Joint-Embedding Predictive Architecture (LeJEPA) for biosignal learning. Pre-trained on over 21,000 hours of unlabeled EEG from the TUEG corpus, LuMamba is evaluated on five downstream tasks spanning abnormality detection, artifact recognition, and mental condition classification across electrode configurations ranging from 16 to 26 channels. In the pre-training objective, masked reconstruction alone yields structured but less generalizable representations, while LeJEPA alone produces diffuse embeddings; combining both objectives achieves the most robust performance. With only 4.6M parameters, LuMamba attains 80.99% balanced accuracy on TUAB and achieves state-of-art performance on Alzheimer's detection (0.97 AUPR), while requiring 377x fewer FLOPS than state-of-art models at equivalent sequence lengths and scaling to 12x longer sequences before reaching typical GPU memory limits. Code is available at https://github.com/pulp-bio/biofoundation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。