用自监督学习构建重症监护时间序列基础模型，小数据下表现更优。
Towards Self-Supervised Foundation Models for Critical Care Time Series
- 基于双轴变换器架构，在多个电子病历数据集上预训练。
- 在小样本（<5000例）死亡率预测任务中超越有监督基线。
- 适合资源有限的临床场景，推动可泛化医疗AI应用。
近年来，医疗领域专用基础模型发展迅速，但针对重症监护时间序列的基础模型仍相对较少，主要受限于数据集规模和可用性。本文提出一种早期预训练的重症监护时间序列基础模型，采用双轴变换器（Bi-Axial Transformer, BAT）架构，并在聚合的电子健康记录数据集上进行训练。通过微调该模型在与训练数据来源不同的数据集上进行死亡率预测，我们验证了其有效的迁移学习能力，尤其在小样本数据集（<5,000例）上显著优于有监督基线模型。这些成果表明，自监督基础模型在重症监护时间序列分析中具有潜力，可支持资源有限环境下通用且稳健的临床应用。
原文摘要 · Abstract (English)
Domain-specific foundation models for healthcare have expanded rapidly in recent years, yet foundation models for critical care time series remain relatively underexplored due to the limited size and availability of datasets. In this work, we introduce an early-stage pre-trained foundation model for critical care time-series based on the Bi-Axial Transformer (BAT), trained on pooled electronic health record datasets. We demonstrate effective transfer learning by fine-tuning the model on a dataset distinct from the training sources for mortality prediction, where it outperforms supervised baselines, particularly for small datasets ($<5,000$). These contributions highlight the potential of self-supervised foundation models for critical care times series to support generalizable and robust clinical applications in resource-limited settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。