解决生理信号缺模问题，让多模态模型在数据不全时仍稳定准确。
PhysioME: A Robust Multimodal Self-Supervised Framework for Physiological Signals with Missing Modalities
- 融合对比学习与掩码预测的自监督方法，适应缺失模态。
- 双路径神经网络捕捉各信号的时间动态特征。
- 重建解码器可恢复缺失模态，支持不完整输入处理。
由于硬件限制或运动伪影，基于生理信号的医疗应用中常出现模态缺失或损坏。然而，现有方法大多假设所有模态均可用，一旦缺失即导致性能显著下降。为此，本文提出PhysioME框架，旨在提升在模态缺失条件下的鲁棒性。该框架采用：(1) 结合对比学习与掩码预测的多模态自监督学习；(2) 针对每种生理信号模态设计的双路径神经网络（Dual-PathNeuroNet），以捕捉其时间动态；(3) 用于重建缺失模态标记的恢复解码器，实现对不完整输入的灵活处理。实验表明，PhysioME在多种缺失场景下均表现出高一致性与泛化能力，展现出在真实临床环境中应对数据不完备问题的可靠性。
原文摘要 · Abstract (English)
Missing or corrupted modalities are common in physiological signal-based medical applications owing to hardware constraints or motion artifacts. However, most existing methods assume the availability of all modalities, resulting in substantial performance degradation in the absence of any modality. To overcome this limitation, this study proposes PhysioME, a robust framework designed to ensure reliable performance under missing modality conditions. PhysioME adopts: (1) a multimodal self-supervised learning approach that combines contrastive learning with masked prediction; (2) a Dual-PathNeuroNet backbone tailored to capture the temporal dynamics of each physiological signal modality; and (3) a restoration decoder that reconstructs missing modality tokens, enabling flexible processing of incomplete inputs. The experimental results show that PhysioME achieves high consistency and generalization performance across various missing modality scenarios. These findings highlight the potential of PhysioME as a reliable tool for supporting clinical decision-making in real-world settings with imperfect data availability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。