arXiv:2604.18570cs.LGcs.AI2026-04被引 3

构建可覆盖全病程的虚拟患者模型,实现跨模态医疗数据统一表示。

A multimodal and temporal foundation model for virtual patient representations at healthcare system scale

论文配图:A multimodal and temporal foundation model for virtual patient representations at healthcare system scale
图 1 · 摘自论文原文
  • 基于30年720万患者数据,融合28类医疗模态与10万+医学事件建模。
  • 提前五年预测新发疾病风险,准确率在95项任务中表现优异。
  • 支持文本与图像混合查询,可用于临床决策辅助与医学检索。

现代医学产生大量跨孤岛系统的多模态数据,但尚无模型能将完整的临床记录及其时间深度整合为统一的患者表征。我们提出Apollo,一个在大型美国医疗系统超过三十年的纵向住院记录上训练和评估的多模态时序基础模型,包含来自720万名患者的250亿条记录,涵盖28种不同医学模态及12个主要专科。Apollo学习了一个统一的表征空间,整合了我们临床词汇中的超10万种独特医学事件,以及图像与临床文本。这一“医学概念图谱”构成建模完整患者诊疗历程的计算基底,将结构化与非结构化事件序列压缩为虚拟患者表征。为评估这些全患者表征的潜力,我们在140万患者的独立测试集中构建了322项预后与检索任务。结果显示,Apollo嵌入具备强大的泛化临床预测能力,包括提前五年预测新发病风险(95项任务)、疾病进展(78项)、治疗反应(59项)、治疗相关不良事件风险(17项)及医院运营指标(12项)。通过特征归因技术,我们证明模型预测与临床可解释的多模态生物标志物一致。在61项检索任务中评估语义相似性搜索,并进一步展示其作为多模态医学搜索引擎的潜力,支持文本与图像查询。这些建模能力共同奠定了可计算医学的基础,使患者照护的全部背景信息可被计算推理所利用。

原文摘要 · Abstract (English)

Modern medicine generates vast multimodal data across siloed systems, yet no existing model integrates the full breadth and temporal depth of the clinical record into a unified patient representation. We introduce Apollo, a multimodal temporal foundation model trained and evaluated on over three decades of longitudinal hospital records from a major US hospital system, composed of 25 billion records from 7.2 million patients, representing 28 distinct medical modalities and 12 major medical specialties. Apollo learns a unified representation space integrating over 100 thousand unique medical events in our clinical vocabulary as well as images and clinical text. This "atlas of medical concepts" forms a computational substrate for modeling entire patient care journeys comprised of sequences of structured and unstructured events, which are compressed by Apollo into virtual patient representations. To assess the potential of these whole-patient representations, we created 322 prognosis and retrieval tasks from a held-out test set of 1.4 million patients. We demonstrate the generalized clinical forecasting potential of Apollo embeddings, including predicting new disease onset risk up to five years in advance (95 tasks), disease progression (78 tasks), treatment response (59 tasks), risk of treatment-related adverse events (17 tasks), and hospital operations endpoints (12 tasks). Using feature attribution techniques, we show that model predictions align with clinically-interpretable multimodal biomarkers. We evaluate semantic similarity search on 61 retrieval tasks, and moreover demonstrate the potential of Apollo as a multimodal medical search engine using text and image queries. Together, these modeling capabilities establish the foundation for computable medicine, where the full context of patient care becomes accessible to computational reasoning.

虚拟患者多模态时序建模临床预测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。