用双曲空间建模脑电多模态数据,提升心理状态识别精度
EEG-Based Multimodal Learning via Hyperbolic Mixture-of-Curvature Experts
- 为不同模态分配可学习曲率的双曲专家,自适应捕捉层次结构
- 在情感识别、睡眠分期等任务上达到当前最优性能
- 适合关注脑机接口与神经信号建模的研究者
基于脑电图(EEG)的多模态学习通过融合脑信号与互补模态,提升心理状态评估能力,具有重要临床价值。该范式的效果主要依赖于异构模态的表示学习。对于EEG相关任务,一个有前景的方向是利用其层次结构特性,因为最新研究表明,脑电和相关模态(如面部表情)均表现出反映复杂认知过程的层次结构。然而,欧氏嵌入因几何平坦难以有效表征此类结构,而双曲空间因其指数增长特性天然适合。本文提出EEG-MoCE:一种新颖的双曲曲率混合专家框架,用于多模态神经技术。该方法将每种模态分配至可学习曲率的双曲空间中的专家,实现对其内在几何结构的自适应建模。随后,采用曲率感知融合策略动态加权专家,突出具有更丰富层次信息的模态。在基准数据集上的大量实验表明,该方法在情绪识别、睡眠分期及认知评估任务中均达到最先进水平。代码已开源:https://github.com/zhourunhe/EEG-MoCE。
原文摘要 · Abstract (English)
Electroencephalography (EEG)-based multimodal learning integrates brain signals with complementary modalities to improve mental state assessment, providing great clinical potential. The effectiveness of such paradigms largely depends on the representation learning on heterogeneous modalities. For EEG-based paradigms, one promising approach is to leverage their hierarchical structures, as recent studies have shown that both EEG and associated modalities (e.g., facial expressions) exhibit hierarchical structures reflecting complex cognitive processes. However, Euclidean embeddings struggle to represent these hierarchical structures due to their flat geometry, while hyperbolic spaces, with their exponential growth property, are naturally suited for them. In this work, we propose EEG-MoCE, a novel hyperbolic mixture-of-curvature experts framework designed for multimodal neurotechnology. EEG-MoCE assigns each modality to an expert in a learnable-curvature hyperbolic space, enabling adaptive modeling of its intrinsic geometry. A curvature-aware fusion strategy then dynamically weights experts, emphasizing modalities with richer hierarchical information. Extensive experiments on benchmark datasets demonstrate that EEG-MoCE achieves state-of-the-art performance, including emotion recognition, sleep staging, and cognitive assessment. Code is available at https://github.com/zhourunhe/EEG-MoCE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。