arXiv:2603.15644cs.LGcs.CL2026-03被引 6

优化电子病历分词方式,提升模型性能与效率

Tokenization Tradeoffs in Structured EHR Foundation Models

  • 联合事件与位置编码将临床事件和时间信息融合为单一标记
  • 在73项任务中表现优于其他方法,预训练计算量减少近40%
  • 效果可跨机构泛化,适合医疗大模型研发人员参考

面向结构化电子健康记录(EHR)的基座模型通过在纵向时间序列临床事件上预训练,学习可迁移的患者表征。分词策略——即如何将时间线转化为离散模型输入——决定了信息保留程度、编码效率以及需由模型学习或预先计算的关系。然而,分词设计对下游性能和计算效率的影响仍不明确。本文在儿科EHR数据上采用因子实验设计,系统调整事件编码、时间编码和工作流标注方式,评估了74项临床预测任务的受试者工作特征曲线下面积(AUC)。结果表明,联合事件编码与位置时间编码在73/74和71/74任务中表现更优,且分别降低39.5%和9.6%的预训练浮点运算量。消融实验显示其优势源于局部绑定效率:代码-属性对被合并为单一标记,而非分散于多个标记,避免模型在预训练阶段学习关联。在成人重症监护室队列上的外部验证表明该优势具有泛化性,尽管存在词汇不匹配;而时间与工作流效应仍具机构特异性。研究确立分词为提升EHR基座模型性能与效率的关键调控变量。

原文摘要 · Abstract (English)

Foundation models for structured electronic health records (EHRs) are pretrained on longitudinal sequences of timestamped clinical events to learn adaptable patient representations. Tokenization -- how these timelines are converted into discrete model inputs -- determines what information is preserved, how efficiently it is encoded, and which relationships must be learned versus precomputed. Yet the impact of tokenization design choices on downstream performance and computational efficiency remains largely unexplored. Here, we pretrained a transformer on pediatric EHR data under a factorial design, varying tokenization along event encoding, time encoding, and workflow annotation. We evaluated area-under-the-receiver-operating-characteristic curve across 74 clinical prediction tasks. Joint event encoding and positional time encoding outperformed their alternatives (73/74 and 71/74 tasks) while requiring 39.5% and 9.6% fewer pretraining floating-point operations, respectively. Targeted ablations traced the joint encoding advantage to local binding efficiency, that is, code-attribute pairs are combined into single tokens, rather than split across tokens that the model must learn to associate during pretraining. External evaluation on an adult intensive care unit cohort demonstrated that this advantage generalizes despite substantial vocabulary mismatch, while temporal and workflow effects remain institution-specific. These results establish tokenization as a tractable lever for improving both the performance and efficiency of EHR foundation models.

电子病历分词优化大模型效率

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。