通过多尺度对齐,让心电图与临床文本更好匹配。
From Token to Rhythm: A Multi-Scale Approach for ECG-Language Pretraining
- 分三个时间尺度(片段、搏动、节律)对齐心电图与文本。
- 在三个公开数据集上实现零样本分类等任务的领先性能。
- 适合医疗AI研究者和需要跨模态预训练的开发者。
心电图(ECG)在心脏健康监测和心脏病诊断中至关重要。传统深度学习方法依赖大量人工标注,耗时且成本高。自监督学习(SSL)为此提供了替代方案,可高效提取可迁移的ECG表征。然而,现有研究常忽视ECG信号的多尺度特性,难以建模其层次结构,限制了泛化能力。为此,我们提出MELP模型,一种多尺度心电图-语言预训练方法,充分利用心电图与临床文本对的层级监督。MELP首先预训练专用于心血管领域的语言模型,提升对医学文本的理解;随后在片段、搏动和节律三个层级施加跨模态监督,实现多尺度对齐。我们在三个公开的ECG数据集上评估了MELP,涵盖零样本分类、线性探测和迁移学习等多种任务。实验结果表明,MELP显著优于现有自监督方法,证明其在多种临床场景中的有效性与适应性。代码已开源:https://github.com/HKU-MedAI/MELP。
原文摘要 · Abstract (English)
Electrocardiograms (ECGs) play a vital role in monitoring cardiac health and diagnosing heart diseases. However, traditional deep learning approaches for ECG analysis rely heavily on large-scale manual annotations, which are both time-consuming and resource-intensive to obtain. To overcome this limitation, self-supervised learning (SSL) has emerged as a promising alternative, enabling the extraction of robust ECG representations that can be efficiently transferred to various downstream tasks. While previous studies have explored SSL for ECG pretraining and multi-modal ECG-language alignment, they often fail to capture the multi-scale nature of ECG signals. As a result, these methods struggle to learn generalized representations due to their inability to model the hierarchical structure of ECG data. To address this gap, we introduce MELP, a novel Multi-scale ECG-Language Pretraining (MELP) model that fully leverages hierarchical supervision from ECG-text pairs. MELP first pretrains a cardiology-specific language model to enhance its understanding of clinical text. It then applies three levels of cross-modal supervision-at the token, beat, and rhythm levels-to align ECG signals with textual reports, capturing structured information across different time scales. We evaluate MELP on three public ECG datasets across multiple tasks, including zero-shot ECG classification, linear probing, and transfer learning. Experimental results demonstrate that MELP outperforms existing SSL methods, underscoring its effectiveness and adaptability across diverse clinical applications. Our code is available at https://github.com/HKU-MedAI/MELP.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。