arXiv:2410.02131cs.LGcs.CL2024-10ICML被引 17

用对比掩码自编码器融合心电图与文本,提升医疗诊断准确性。

Boosting Masked ECG-Text Auto-Encoders as Discriminative Learners

  • 通过掩码模态建模和改进负采样,实现跨模态对齐。
  • 仅用1%数据训练时AUC平均提升15%,零样本下提升2%。
  • 适合医疗多模态学习、小样本诊断模型研究者使用。

准确解读心电图(ECG)信号对心血管疾病诊断至关重要。将ECG信号与伴随的文本报告结合,有望通过融合生理数据与定性见解,进一步提升临床诊断能力。然而,由于模态间固有差异以及标注数据稀缺,跨模态学习面临巨大挑战。为此,我们提出D-BETA框架,采用对比掩码自编码器预训练ECG与文本数据。D-BETA创新性地融合生成式与增强判别能力,通过掩码模态建模、专用损失函数及针对跨模态对齐优化的负采样策略,实现鲁棒的跨模态表示。在五个公开数据集上的大量实验表明,该方法显著优于现有技术:在线性探针任务中,仅使用1%训练数据时平均AUC提升15%;在无需训练数据的零样本场景下,性能提升2%。结果验证了D-BETA的有效性,凸显其在自动化临床诊断中的潜力。代码与模型检查点已开源。

原文摘要 · Abstract (English)

The accurate interpretation of Electrocardiogram (ECG) signals is pivotal for diagnosing cardiovascular diseases. Integrating ECG signals with accompanying textual reports further holds immense potential to enhance clinical diagnostics by combining physiological data and qualitative insights. However, this integration faces significant challenges due to inherent modality disparities and the scarcity of labeled data for robust cross-modal learning. To address these obstacles, we propose D-BETA, a novel framework that pre-trains ECG and text data using a contrastive masked auto-encoder architecture. D-BETA uniquely combines the strengths of generative with boosted discriminative capabilities to achieve robust cross-modal representations. This is accomplished through masked modality modeling, specialized loss functions, and an improved negative sampling strategy tailored for cross-modal alignment. Extensive experiments on five public datasets across diverse downstream tasks demonstrate that D-BETA significantly outperforms existing methods, achieving an average AUC improvement of 15% in linear probing with only one percent of training data and 2% in zero-shot performance without requiring training data over state-of-the-art models. These results highlight the effectiveness of D-BETA, underscoring its potential to advance automated clinical diagnostics through multi-modal representations. Our sample code and checkpoint are made available at https://github.com/manhph2211/D-BETA.

医疗AI多模态自编码器小样本

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。