将化验结果转为可解释的离散令牌,提升医疗预测模型的可解释性与性能。
Explainable Transformer Models for Clinical Prediction Tasks on Structured Electronic Health Records

- 用百分位分箱法将化验值编码为离散令牌,保留数值等级信息
- 在7500万患者数据上预训练,化验相关任务表现优于现有基准模型
- 集成梯度生成逐令牌归因,结果符合临床已知风险因素
面向结构化电子健康记录(EHR)的预测模型在医疗人工智能中仍具核心地位,但多数未兼顾定量实验室信息与输入医学事件的可解释性。本文提出BERT-LER,一种基于BERT架构的模型,在7500万匿名患者EHR数据上预训练并微调,将实验室检查结果以离散令牌形式编码,同时通过百分位分箱保留其分级信息,并结合集成梯度实现基于原始EHR序列的令牌级归因。我们在公开的EHRShot基准套件和基于真实世界数据的哮喘严重程度进展研究中评估该方法。BERT-LER在多项任务中表现媲美甚至超越现有公开基准模型,尤其在实验室相关任务上优势明显,且归因结果与临床已知风险因素一致。该框架统一了实验室值表示与可解释性,适用于多种治疗领域与预测任务。
原文摘要 · Abstract (English)
Predictive models over structured electronic health records (EHRs) remain central to machine learning for healthcare, but few have jointly emphasized quantitative laboratory information and interpretability with respect to input medical events. We present BERT-LER, a BERT-style model for coded EHR timelines pretrained and fine-tuned from a de-identified EHR dataset of 75 million patients, that encodes laboratory test results as discrete tokens while retaining graded information through percentile-based binning, paired with Integrated Gradients for token-level attributions grounded in the input EHR sequence. We benchmark our approach on the public EHRShot benchmark suite and on an asthma severity progression study based on real-world data. This addresses a methodological gap in EHR foundation-style modeling by unifying laboratory value representation and explainability in a single framework, while assessing whether both predictive performance and explanations generalize beyond standard clinical prediction tasks. Across EHRShot and asthma tasks, BERT-LER achieves predictive performance that is competitive with, and on laboratory-related tasks often exceeds, publicly available benchmark models, and provides attributions that align with clinically known risk factors. Our architecture and explainability approach can be applied to many therapeutic areas and prediction tasks using language models trained on structured EHRs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。