用合成数据高效预训练时间序列模型，效果不输真实数据。
CauKer: Classification Time Series Foundation Models Can Be Pretrained on Synthetic Data
- 结合高斯过程与因果模型生成有真实趋势的合成时序数据。
- 在10K到10M样本间展现清晰的规模效应，真实数据则无规律。
- 适合需要快速构建模型且缺乏标注数据的研究者。
时间序列基础模型（TSFMs）因零样本能力强大而备受关注，但通常需在大规模真实序列上进行计算密集型预训练。为实现样本高效的预训练，本文提出新算法CauKer，通过组合高斯过程核与结构因果模型，生成具有真实趋势、季节性和非线性关系的多样化合成时序数据。该方法适用于不同架构和预训练策略的先进分类TSFMs。实验表明，由CauKer生成的数据集在10K至10M样本量范围内，以及模型参数从100万到7.83亿之间均呈现清晰的规模规律，而真实数据集则表现出不规则的缩放行为。代码已开源。
原文摘要 · Abstract (English)
Time series foundation models (TSFMs) have recently gained significant attention due to their strong zero-shot capabilities and widespread real-world applications. Such models typically require a computationally costly pre-training on large-scale, carefully curated collections of real-world sequences. To allow for a sample-efficient pre-training of TSFMs, we propose \textsc{CauKer}, a novel algorithm designed to generate diverse, causally coherent synthetic time series with realistic trends, seasonality, and nonlinear interactions. \textsc{CauKer} combines Gaussian Process (GP) kernel composition with Structural Causal Models (SCM) to produce data for sample-efficient pre-training of state-of-the-art classification TSFMs having different architectures and following different pre-training approaches. Additionally, our experiments reveal that \textsc{CauKer}-generated datasets exhibit clear scaling laws for both dataset size (10K to 10M samples) and model capacity (1M to 783M parameters), unlike real-world datasets, which display irregular scaling behavior. The source code is publicly available at https://github.com/ShifengXIE/CauKer.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。