arXiv:2409.00122eess.SPcs.AI2024-09KDD被引 33

用脑电图知识提升其他生理信号分析,实现跨模态协同优化。

Brant-X: A Unified Physiological Signal Alignment Framework

  • 基于脑电基础模型,高效迁移脑电信号知识到其他生理信号
  • 设计双层级对齐机制,统一不同语义层次的信号表征
  • 在睡眠分期、情绪识别等任务中达到顶尖效果,适合多模态生理分析

生理信号是理解人体生理状态的关键线索。现有研究多聚焦单一信号类型,但人体是一个整体系统,各类生理数据间存在内在关联。尤其脑电图(EEG)作为生命活动控制中心,与其它信号密切相关,其相关性可提升多种场景下的性能。然而,受限于同步采集数据稀缺、信号间相关性差异及任务间相关性变化,这一目标仍具挑战。为此,我们提出统一的生理信号对齐框架 Brant-X,通过(1)利用脑电基础模型,以数据高效方式将丰富的脑电知识迁移到其他生理信号;(2)引入双层级对齐机制,从不同语义尺度上充分对齐脑电与其他信号的语义。实验表明,Brant-X 在睡眠阶段分类、情绪识别、冻结步态检测和眼动通信等多种下游任务中,均优于任务无关与任务特定基线方法。对心律失常检测任务的分析与案例可视化进一步验证了从脑电到其他信号的知识迁移有效性。模型主页:https://github.com/zjunet/Brant-X/

原文摘要 · Abstract (English)

Physiological signals serve as indispensable clues for understanding various physiological states of human bodies. Most existing works have focused on a single type of physiological signals for a range of application scenarios. However, as the body is a holistic biological system, the inherent interconnection among various physiological data should not be neglected. In particular, given the brain's role as the control center for vital activities, electroencephalogram (EEG) exhibits significant correlations with other physiological signals. Therefore, the correlation between EEG and other physiological signals holds potential to improve performance in various scenarios. Nevertheless, achieving this goal is still constrained by several challenges: the scarcity of simultaneously collected physiological data, the differences in correlations between various signals, and the correlation differences between various tasks. To address these issues, we propose a unified physiological signal alignment framework, Brant-X, to model the correlation between EEG and other signals. Our approach (1) employs the EEG foundation model to data-efficiently transfer the rich knowledge in EEG to other physiological signals, and (2) introduces the two-level alignment to fully align the semantics of EEG and other signals from different semantic scales. In the experiments, Brant-X achieves state-of-the-art performance compared with task-agnostic and task-specific baselines on various downstream tasks in diverse scenarios, including sleep stage classification, emotion recognition, freezing of gaits detection, and eye movement communication. Moreover, the analysis on the arrhythmia detection task and the visualization in case study further illustrate the effectiveness of Brant-X in the knowledge transfer from EEG to other physiological signals. The model's homepage is at https://github.com/zjunet/Brant-X/.

生理信号脑电图知识迁移多模态对齐

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。