arXiv:2603.24562cs.LG2026-03被引 2

用复发感知的下一次就诊预测,让医疗记录大模型更懂病人病情演变。

Scaling Recurrence-aware Foundation Models for Clinical Records via Next-Visit Prediction

  • 基于复发感知的事件预测,让模型学会生成患者下一次就诊的临床事件序列。
  • 在百万级患者数据上,零样本预测多种疾病发病率,性能媲美微调模型。
  • 揭示了重复事件混淆导致评估虚高,适合医疗时序建模与跨机构泛化研究者。

尽管大规模预训练已革新语言建模,其在结构化电子健康记录(EHR)中的潜力仍待挖掘。我们提出RAVEN,一种基于复发感知下一次就诊事件预测的生成式预训练策略。利用超过一百万独特个体的数据集,模型可基于患者历史自回归生成下一次就诊的编码临床事件。我们引入对重复事件的正则化,并指出一个关键问题:在基于EHR的基础模型评估中,若未区分新发事件与后续复发事件,重复事件标记会人为抬高性能指标。此外,我们在数据受限、算力饱和的场景下实证考察了模型扩展行为,发现仅增大模型规模而不相应增加数据量是次优的。通过零样本预测多种疾病的发病情况,RAVEN的表现可媲美全微调的基于表示的Transformer模型,优于标准的基于模拟的下一步词预测方法和提示式医学大语言模型基线。最后,在不进行额外参数更新的情况下,RAVEN能在存在临床代码映射失真和特征覆盖缺口的外部患者队列上实现泛化。

原文摘要 · Abstract (English)

While large-scale pretraining has revolutionized language modeling, its potential remains underexplored in healthcare with structured electronic health records (EHRs). We present RAVEN, a novel generative pretraining strategy for sequential EHR data based on Recurrence-Aware next-Visit EveNt prediction. Leveraging a dataset of over one million unique individuals, our model learns to autoregressively generate tokenized clinical events for the next visit conditioned on patient history. We introduce regularization on predicting repeated events and highlight a key pitfall in EHR-based foundation model evaluations: repeated event tokens can inflate performance metrics when new onsets are not distinguished from subsequent occurrences. Furthermore, we empirically investigate the scaling behaviors in a data-constrained, compute-saturated regime, showing that simply increasing model size is suboptimal without commensurate increases in data volume. We evaluate our model via zero-shot prediction for forecasting the incidence of a diverse set of diseases, where it rivals fully fine-tuned representation-based Transformer models and outperforms both standard simulation-based next-token approaches and a prompted medical large language model baseline. Finally, without additional parameter updates, we show that RAVEN can generalize to an external patient cohort under lossy clinical code mappings and feature coverage gaps.

医疗AI时序建模大模型EHR

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。