将心电图与病历数据融合，用大模型精准预测临床结局
UniPACT: A Multimodal Framework for Prognostic Question Answering on Raw ECG and Structured EHR
- 把数值型病历转为语义文本，与原始心电图特征联合输入大模型
- 在多个预后任务上达到89.37%的平均AUROC，优于专用模型
- 适合临床决策支持系统研发者及多模态医疗AI研究者
准确的临床预后需融合结构化电子健康记录（EHR）与实时生理信号（如心电图ECG）。大语言模型（LLMs）虽具备强大推理能力，却难以原生处理异构非文本数据。为此，我们提出UniPACT（统一临床时间序列预后问答框架），通过结构化提示机制将数值型EHR转化为语义丰富文本，并与直接从原始ECG波形学习到的表征融合，使LLM能跨模态综合推理。我们在MDS-ED基准上评估该方法，实现89.37%的平均AUROC，涵盖诊断、恶化、ICU入院和死亡等多样预后任务，超越专用基线模型。进一步分析表明，多模态多任务设计对性能至关重要，且在缺失数据场景下表现稳健。
原文摘要 · Abstract (English)
Accurate clinical prognosis requires synthesizing structured Electronic Health Records (EHRs) with real-time physiological signals like the Electrocardiogram (ECG). Large Language Models (LLMs) offer a powerful reasoning engine for this task but struggle to natively process these heterogeneous, non-textual data types. To address this, we propose UniPACT (Unified Prognostic Question Answering for Clinical Time-series), a unified framework for prognostic question answering that bridges this modality gap. UniPACT's core contribution is a structured prompting mechanism that converts numerical EHR data into semantically rich text. This textualized patient context is then fused with representations learned directly from raw ECG waveforms, enabling an LLM to reason over both modalities holistically. We evaluate UniPACT on the comprehensive MDS-ED benchmark, it achieves a state-of-the-art mean AUROC of 89.37% across a diverse set of prognostic tasks including diagnosis, deterioration, ICU admission, and mortality, outperforming specialized baselines. Further analysis demonstrates that our multimodal, multi-task approach is critical for performance and provides robustness in missing data scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。