将ICD编码的层级结构融入电子病历建模,提升临床表征学习效果。
Hierarchical Modeling of ICD Codes in EHR Foundation Models

- 在Transformer中加入不同层级的ICD编码作为额外输入 token
- 层级编码显著提升跨数据集和本域任务的预测性能
- 适用于需要精准疾病表征的医疗AI研究者
电子病历基础模型通常将ICD诊断码视为扁平标记,忽略了其蕴含的疾病家族、子类与细粒度诊断信息的临床层级结构。本文以ICD-10-CM层级为归纳偏置,研究两种互补机制:一是在BERT式Transformer中引入多层级ICD编码作为上下文;二是通过层级感知边结合诊断共现结构,构建图模型中的代码表示。在大规模真实临床数据集MIMIC-IV(预训练与本域评估)和eICU(跨数据集迁移探测)上验证发现,显式编码层级结构在本域与跨域设置下均优于扁平表示,且最有效的层级因任务与建模方式而异。结果表明,层级感知的EHR表示学习具有广泛适用性。
原文摘要 · Abstract (English)
Electronic health record foundation models typically treat ICD diagnosis codes as flat tokens, overlooking the clinically meaningful hierarchical structure that captures disease families, subcategories, and fine-grained diagnostic detail. As a result, existing EHR representation learning methods do not explicitly exploit the hierarchical structure already present in the coding system. In this work, we study ICD-10-CM hierarchy as a general inductive bias for clinical representation learning. We investigate two complementary mechanisms for incorporating hierarchy: first, by augmenting diagnosis sequences in a BERT-style transformer with tokens corresponding to different levels of the ICD hierarchy, and second, by injecting hierarchy into graph-based code representations through hierarchy-aware edges combined with diagnosis co-occurrence structure. Across these settings, we evaluate whether explicit hierarchy improves downstream prediction, which levels of the hierarchy are most useful, whether hierarchy encoding improves transfer across datasets, and how hierarchy reshapes embedding similarity structure. We conduct experiments on two large-scale real-world clinical datasets: MIMIC-IV, used for pretraining and in-domain evaluation, and eICU, used to assess cross-dataset transfer via frozen encoder probing. Our findings show that explicitly encoding ICD hierarchy improves over flat code representations in both in-domain and cross-dataset settings, while revealing that the most useful level of hierarchy depends on both the task and the modeling approach. More broadly, we focus on hierarchy-aware EHR representation learning and show that the benefits of encoding hierarchy are generalizable across modeling settings and hierarchy levels.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。