将心电图转为离散令牌，让大模型能像理解文字一样分析心电图。
HeartLLM: Discretized ECG Tokenization for LLM-Based Diagnostic Reasoning
- 用分导联编码和量化模块把连续心电图转成离散代码
- 在心电图问答和报告生成任务上表现优异，且对新数据泛化能力强
- 适合医疗AI研究者和想用大模型做临床推理的开发者
心电图（ECG）在心血管诊断中至关重要，但现有自动化方法往往难以跨任务泛化，且不支持开放式推理。我们提出HeartLLM，一种新框架，通过将12导联心电图信号转化为可被大语言模型（LLM）处理的离散令牌，实现心电图与自然语言的统一建模。该方法使用分导联编码器和量化模块将连续心电图嵌入转化为量化代码，并映射到扩展的心电图词汇表形成心电图令牌。为弥合模态差距，模型在自回归心电图令牌预测任务上预训练，使LLM借助语言建模能力捕捉时间动态。最后在心电图问答和诊断报告生成任务上进行指令微调。无需修改核心模型，HeartLLM在多个任务上表现强劲，并保持对分布外数据的良好泛化能力。大量实验验证了各组件有效性，展示了将离散心电图令牌融入大模型在医学推理中的潜力。
原文摘要 · Abstract (English)
Electrocardiography (ECG) plays a central role in cardiovascular diagnostics, yet existing automated approaches often struggle to generalize across clinical tasks and offer limited support for open-ended reasoning. We present HeartLLM, a novel framework that integrates time-series (TS) and language modeling by enabling large language models (LLMs) to process 12-lead ECG signals for clinical text generation tasks. Our approach discretizes continuous ECG embeddings into quantized codes using a lead-wise encoder and quantization module. These quantized codes are then mapped to an extended ECG vocabulary to form ECG tokens, enabling the model to process both ECG and natural language inputs within a unified framework. To bridge the modality gap, we pretrain the model on an autoregressive ECG token forecasting task, allowing the LLM to capture temporal dynamics through its inherent language modeling capability. Finally, we perform instruction tuning on both ECG question answering and diagnostic report generation. Without modifying the core model, HeartLLM achieves strong performance across tasks while maintaining generalization to out-of-distribution settings. Extensive experiments demonstrate the effectiveness of each component and highlight the potential of integrating discretized ECG tokens into LLMs for medical reasoning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。