通过跨模态时序重建,让生物信号学习更符合生理节奏的表示。
Physiology-Aware Masked Cross-Modal Reconstruction for Biosignal Representation Learning

- 用时序有序的多模态信号做掩码重建,强制学习生理时间结构。
- 在19项任务中15项超越基线,尤其在心电与脉搏信号分析上表现优异。
- 适合关注生理信号建模、医疗健康分析的研究者使用。
从身体不同部位获取的生物信号通常以时间顺序反映同一生理过程。然而,现有自监督学习方法将这些信号视为可互换视图,忽略了其内在的时间方向性。以心电图(ECG)和光电容积脉搏波(PPG)为例,前者记录每次心跳的电激活,后者则反映因血管动力学延迟后的外周脉搏。为此,我们提出xMAE,一种基于时序有序生物信号的掩码跨模态重建预训练框架,利用时间约束促进学习到的表征具备生理意义的时间结构。实验表明,xMAE预训练的表征在19项下游任务中的15项上优于单模态与多模态基线,涵盖心血管结局预测、异常化验检测、睡眠分期及人口统计推断,且在不同设备、体位和采集条件下均具有良好泛化能力。进一步分析显示,学习到的PPG表征中包含了ECG-PPG的时间结构信息。这证明当多模态信号处于同一生理过程的不同阶段时,引入时间结构能显著提升多模态预训练效果。代码已开源:https://github.com/hzhou3/xMAE。
原文摘要 · Abstract (English)
Biosignals acquired from different locations on the body often provide temporally ordered views of the same underlying physiological process. However, most existing self supervised learning methods treat these signals as interchangeable views, overlooking the directional temporal dynamics that link them. A canonical example is the relationship between electrocardiography (ECG), which captures the electrical activation initiating each heartbeat, and photoplethysmography (PPG), which records the resulting peripheral pulse delayed by vascular dynamics. To capture this structured relationship, we introduce xMAE, a biosignal pretraining framework that leverages masked cross modal reconstruction across temporally ordered biosignals as a training time constraint to encourage physiologically meaningful timing structure in the learned representations. We show that pretraining with xMAE yields representations that outperform both unimodal and multimodal baselines on 15 of 19 downstream tasks, including cardiovascular outcome prediction, abnormal laboratory test detection, sleep staging, and demographic inference, while generalizing across devices, body locations, and acquisition settings. Further analysis suggests that the ECG PPG timing structure is reflected in the learned PPG representations. More broadly, xMAE demonstrates the effectiveness of incorporating temporal structure into multimodal pretraining when signals observe different stages of a shared underlying process. Code is available at https://github.com/hzhou3/xMAE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。