用大模型统一处理病历文本和结构化数据,提升疾病分类准确率。
Primary ICD Category Prediction using LLM-based Probing

- 冻结大模型提取多模态嵌入,用线性探测器统一预测诊断类别
- 联合输入达到87.69%严格准确率,优于单一模态与基线方法
- 小参数适配器可实现跨数据集迁移,仅需5%标注数据
ICD编码在医保报销、研究和人群健康监测中至关重要,但现有自动化系统难以融合临床文本与结构化电子健康记录(EHR)信息。本文评估了冻结的医学大语言模型(LLM)表示是否可作为多模态主诊断类别预测的共享嵌入空间。基于MIMIC-IV数据集,构建包含13,645例住院患者、涵盖10个最常见主要ICD-10代码并合并为7类的队列。将结构化变量序列化为临床叙述,并与去泄漏的出院记录结合。采用冻结的MedFound-Llama3-8B微调骨干网络,从五个Transformer层提取隐藏状态,训练仅使用结构化、仅使用非结构化及联合输入的线性探测器,对比XGBoost与信息匹配的PLM-ICD基线,并测试通过紧凑瓶颈适配器在MIMIC-III上的迁移能力。结果显示,联合探测器在MIMIC-IV上表现最佳(严格准确率87.69%,医学准确率91.45%),超越单模态探测器与基线。结构化探测器比标准基线提升6.19个百分点医学准确率。深层特征呈现更强线性可分性,200万参数适配器仅用5%目标标签即恢复对MIMIC-III的跨数据集迁移能力。结论表明,冻结医学LLM的多模态探测能有效整合结构化与文本数据,支持临床表示在不同模态与数据集间的高效复用。
原文摘要 · Abstract (English)
Objective: ICD codes are central to reimbursement, research, and population health surveillance, yet automated coding systems often struggle to integrate diagnostic signals from both clinical narratives and structured electronic health record (EHR) variables. We evaluated whether frozen medical large language model (LLM) representations can serve as a shared embedding space for multimodal primary diagnosis category prediction. Materials and Methods: We constructed a MIMIC-IV cohort of 13,645 admissions from the 10 most frequent primary ICD-10 codes, consolidated into seven categories. Structured variables were serialized into clinical narratives and combined with leakage-pruned discharge notes. Using a frozen MedFound-Llama3-8B-finetuned backbone, we extracted hidden states from five transformer layers and trained linear probes for structured-only, unstructured-only, and combined inputs, comparing against XGBoost and information-matched PLM-ICD baselines and evaluating MIMIC-III adaptation with a compact bottleneck adapter. Results: The combined probe performed best on MIMIC-IV (87.69% strict; 91.45% medical accuracy), exceeding both single-modality probes and baselines. The structured-only probe outperformed its standard baseline by 6.19 points in medical accuracy. Diagnostic information became increasingly linearly separable in deeper layers, and a 2M-parameter adapter restored cross-dataset transfer to MIMIC-III using only 5% of target labels. Discussion: LLM embeddings can unify structured and narrative EHR information for multimodal diagnosis prediction, supporting efficient reuse of clinical representations across modalities and datasets through a small representation-level module. Conclusion: Multimodal probing of frozen medical LLM representations provides a practical approach for studying EHR modalities and adapting clinical representations across datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。