构建大规模睡眠模型预训练框架，提升跨设备泛化能力
OSF: On Pre-training and Scaling of Sleep Foundation Models
- 基于16.6万小时数据设计通道不变特征学习策略
- 模型规模与多源数据混合显著提升下游任务表现
- 适合睡眠生理分析、临床疾病预测的研究者使用
多导睡眠图（PSG）是睡眠评估的金标准，但不同设备和人群间存在显著异质性。尽管已有研究尝试构建通用睡眠基础模型（FMs），但对预训练过程与扩展规律的理解仍不足。为此，我们从九个公开来源收集了总计166,500小时的睡眠记录，建立SleepBench——一个全面且完全开源的基准测试平台。利用该平台，我们系统评估了四种自监督预训练目标，发现三个关键结论：(1) 现有模型在推理时无法处理缺失通道；(2) 通道不变特征学习对预训练至关重要；(3) 扩大样本量、模型容量及多源数据混合能持续提升下游性能。基于优化后的预训练与扩展方案，我们提出OSF系列睡眠基础模型，在九个数据集上实现多种睡眠与疾病预测任务的当前最优表现。对OSF的进一步分析揭示其在样本效率、层级聚合和跨数据集扩展方面的独特特性。
原文摘要 · Abstract (English)
Polysomnography (PSG) provides the gold standard for sleep assessment but suffers from substantial heterogeneity across recording devices and cohorts. There have been growing efforts to build general-purpose foundation models (FMs) for sleep physiology, but lack an in-depth understanding of the pre-training process and scaling patterns that lead to more generalizable sleep FMs. To fill this gap, we curate a massive corpus of 166,500 hours of sleep recordings from nine public sources and establish SleepBench, a comprehensive, fully open-source benchmark. Leveraging SleepBench, we systematically evaluate four families of self-supervised pre-training objectives and uncover three critical findings: (1) existing FMs fail to generalize to missing channels at inference; (2) channel-invariant feature learning is essential for pre-training; and (3) scaling sample size, model capacity, and multi-source data mixture consistently improves downstream performance.With an enhanced pre-training and scaling recipe, we introduce OSF, a family of sleep FMs that achieves state-of-the-art performance across nine datasets on diverse sleep and disease prediction tasks. Further analysis of OSF also reveals intriguing properties in sample efficiency, hierarchical aggregation, and cross-dataset scaling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。