用高效微调让大模型精准读懂心电图，识别70多种心脏异常。
High-Accuracy ECG Image Interpretation using Parameter-Efficient LoRA Fine-Tuning with Multimodal LLaMA 3.2
- 用低秩适配（LoRA）只改少量参数，高效微调多模态大模型。
- 在PTB-XL数据集上识别超70种心脏异常，精度媲美甚至超越传统CNN。
- 适合医疗AI研究者和临床辅助诊断系统开发者。
心电图（ECG）分析是心脏病诊断的核心。本文提出一种实用方法，利用多模态LLaMA 3.2模型提升心电图图像解读能力。采用参数高效的低秩适配（LoRA）策略，专门增强模型对心电图图像的理解，覆盖多种心脏疾病。方法基于包含100万样本的ECGInstruct指令数据集，该数据集由MIMIC-IV ECG和PTB-XL等开源数据生成，每张心电图配有专家设计的问题与详细解答，涵盖心肌梗死、传导障碍等复杂情形。通过仅更新少量参数（忽略`lm_head`和`embed_tokens`层），高效微调基于LLaMA 3的模型。实验全面验证了方法的有效性，结果表明该方法在多种心电图任务中表现优异，识别超过70种心脏异常的准确率显著优于基线模型，达到或超过传统卷积神经网络（CNN）水平。
原文摘要 · Abstract (English)
Electrocardiogram (ECG) interpretation is a cornerstone of cardiac diagnostics. This paper explores a practical approach to enhance ECG image interpretation using the multimodal LLaMA 3.2 model. We used a parameter-efficient fine-tuning strategy, Low-Rank Adaptation (LoRA), specifically designed to boost the model's ability to understand ECG images and achieve better outcomes across a wide range of cardiac conditions. Our method is tailored for ECG analysis and leverages ECGInstruct, a large-scale instruction dataset with 1 Million samples. This dataset is a rich collection of synthesized ECG images, generated from raw ECG data from trusted open-source repositories like MIMIC-IV ECG and PTB-XL. Each ECG image in ECGInstruct comes with expert-written questions and detailed answers, covering diverse ECG interpretation scenarios, including complex cardiac conditions like Myocardial Infarction and Conduction Disturbances. Our fine-tuning approach efficiently adapts the LLaMA 3.2 model (built upon LLaMA 3) by integrating low-rank adaptation techniques, focusing on efficiency by updating only a small set of parameters, specifically ignoring the `lm_head` and `embed_tokens` layers. This paper details the model setup, our efficient fine-tuning method, and implementation specifics. We provide a thorough evaluation through extensive experiments, demonstrating the effectiveness of our method across various ECG interpretation tasks. The results convincingly show that our parameter-efficient LoRA fine-tuning achieves excellent performance in ECG image interpretation, significantly outperforming baseline models and reaching accuracy comparable to or exceeding traditional CNN-based methods in identifying a wide range of cardiac abnormalities, including over 70 conditions from the PTB-XL dataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。