首次发现电子病历模型遵循类似大模型的缩放规律，可预测性能提升。
Exploring Scaling Laws for EHR Foundation Models
- 在MIMIC-IV数据上训练Transformer，研究模型规模与算力的关系
- 发现算力、参数量、数据量与临床效果呈幂律关系，存在等算力曲线
- 为高效训练医疗大模型提供实证依据，适合临床预测研究者参考
大规模语言模型的缩放定律已深刻影响其发展，使模型规模、数据量和计算资源的系统性增长可预测地带来性能提升。然而，这一原理在电子健康记录（EHR）领域仍缺乏探索——这类数据具有序列性、全球丰富性，但结构不同于自然语言。本文首次对EHR基础模型的缩放规律进行实证研究。通过在MIMIC-IV数据库的患者时间序列数据上，训练不同规模的Transformer模型并覆盖多种计算预算，我们发现了稳定的缩放模式，包括抛物线形的等算力（IsoFLOPs）曲线，以及算力、模型参数、数据规模与临床效用之间的幂律关系。结果表明，EHR模型展现出与大语言模型类似的缩放行为，可为资源高效的训练策略提供预测性指导。本研究为构建强大的EHR基础模型奠定基础，有望推动临床预测任务与个性化医疗的发展。
原文摘要 · Abstract (English)
The emergence of scaling laws has profoundly shaped the development of large language models (LLMs), enabling predictable performance gains through systematic increases in model size, dataset volume, and compute. Yet, these principles remain largely unexplored in the context of electronic health records (EHRs) -- a rich, sequential, and globally abundant data source that differs structurally from natural language. In this work, we present the first empirical investigation of scaling laws for EHR foundation models. By training transformer architectures on patient timeline data from the MIMIC-IV database across varying model sizes and compute budgets, we identify consistent scaling patterns, including parabolic IsoFLOPs curves and power-law relationships between compute, model parameters, data size, and clinical utility. These findings demonstrate that EHR models exhibit scaling behavior analogous to LLMs, offering predictive insights into resource-efficient training strategies. Our results lay the groundwork for developing powerful EHR foundation models capable of transforming clinical prediction tasks and advancing personalized healthcare.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。