用不规则采样数据训练生成模型，发现其只能局部拟合病程，难保临床逻辑一致性。
Limits of Generative Pre-Training in Structured EMR Trajectories with Irregular Sampling
- 在艾滋病与低血压患者病程数据上，用LSTM和简化Transformer做自回归预训练。
- 模型能复现单个特征分布，但跨特征关联结构严重失真。
- 适合关注医疗轨迹生成可靠性的研究者，提醒避免盲目使用通用大模型。
基础模型指通过自然语言处理中的自回归预训练，在大规模数据上训练出的架构，旨在捕捉复杂模式并迁移到下游预测任务。然而，近期一些研究将此类模型用于表型发现而缺乏严格验证，存在生成结果看似合理但临床上不一致的风险。为检验这一偏差，我们在长期抗逆转录病毒治疗（ART）的艾滋病和急性低血压数据集上，训练了序列到序列的LSTM和简化Transformer模型，并在训练中引入随机访视间隔以模拟不规则采样，测试时则保持序列完整。通过评估生成轨迹的分布与相关性保真度，发现两种模型均能复现特征分布，但无法维持跨特征间的结构关系。这表明生成预训练虽能实现局部真实性，却难以保证临床连贯性。该结果强调需采用领域特定评估方法，并支持以轨迹生成作为微调或部署前的有效探针。
原文摘要 · Abstract (English)
Foundation models refer to architectures trained on vast datasets using autoregressive pre-training from natural language processing to capture intricate patterns and motifs. They were originally developed to transfer such learned knowledge to downstream predictive tasks. Recently, however, some studies repurpose these learned representations for phenotype discovery without rigorous validation, risking superficially realistic but clinically incoherent embeddings. To test this mismatch, we trained two autoregressive models -- a sequence-to-sequence LSTM and a reduced Transformer -- on longitudinal ART for HIV and Acute Hypotension datasets. Controlled irregularity was added during training via random inter-visit gaps, while test sequences stayed complete. Patient-trajectory synthesis evaluated distributional and correlational fidelity. Both reproduced feature distributions but failed to preserve cross-feature structure -- showing that generative pre-training yields local realism but limited clinical coherence. These results highlight the need for domain-specific evaluation and support trajectory synthesis as a practical probe before fine-tuning or deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。