arXiv:2601.22128cs.AIcs.CE2026-01被引 3

用动态模拟取代文本预测,让模型学会跟踪患者病程变化。

The Patient is not a Moving Document: A World Model Training Paradigm for Longitudinal EHR

  • 用联合嵌入预测+文本生成双目标训练,让模型提前学习疾病演化规律。
  • 在两组大型队列中验证,能捕捉到传统模型无法识别的疾病动态特征。
  • 适合研究慢性病、个性化治疗等需要长期轨迹建模的临床场景。

以自回归方式训练的大语言模型在生物医学任务中表现优异,其表征通过线性探测在多种任务中取得良好效果,表明患者语义可通过大规模文本预测生成。然而,这种范式将患者视为待总结的文档,而非随干预和时间演化的动态系统。为此,我们提出SMB-Structure,一种基于结构化电子病历的世界模型,结合联合嵌入预测架构(JEPA)与下一个词预测(SFT)。SFT使模型在标记空间重建未来患者状态,而JEPA仅从初始表征出发,在潜在空间预测未来状态,强制模型在观察下一状态前就编码演化动力学。我们在两个大规模队列中验证:纪念斯隆-凯特琳医院(23,319名肿瘤患者;超32.3万患者年)和INSPECT(19,402名肺栓塞患者)。通过在疾病轨迹多个时间点进行线性探测,结果表明该训练范式学习到的嵌入能捕捉到自回归基线无法恢复的疾病动态,使SMB-Structure在高患者异质性的复杂任务中表现优异。模型权重已公开于https://huggingface.co/standardmodelbio/SMB-v1-1.7B-Structure。

原文摘要 · Abstract (English)

Large language models (LLMs) trained with next-word-prediction have achieved success as clinical foundation models. Representations from these language backbones yield strong linear probe performance across biomedical tasks, suggesting that patient semantics emerge from next-token prediction at scale. However, this paradigm treats patients as a document to be summarized rather than a dynamical system to be simulated; a patient's trajectory emerges from their state evolving under interventions and time, requiring models that simulate dynamics rather than predict tokens. To address this, we introduce SMB-Structure, a world model for structured EHR that grounds a joint-embedding prediction architecture (JEPA) with next-token prediction (SFT). SFT grounds our model to reconstruct future patient states in token space, while JEPA predicts those futures in latent space from the initial patient representation alone, forcing trajectory dynamics to be encoded before the next state is observed. We validate across two large-scale cohorts: Memorial Sloan Kettering (23,319 oncology patients; 323,000+ patient-years) and INSPECT (19,402 pulmonary embolism patients). Using a linear probe evaluated at multiple points along the disease trajectory, we demonstrate that our training paradigm learns embeddings that capture disease dynamics not recoverable by autoregressive baselines, enabling SMB-Structure to achieve competitive performance on complex tasks characterized by high patient heterogeneity. Model weights are available at https://huggingface.co/standardmodelbio/SMB-v1-1.7B-Structure.

电子病历世界模型动态建模医疗AI

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。