arXiv:2502.00816cs.LG2025-02ICML被引 165

Sundial用流匹配实现时间序列原生预训练,零样本预测毫秒级响应。

Sundial: A Family of Highly Capable Time Series Foundation Models

  • 提出TimeFlow损失,无需离散化即可对连续时间序列直接建模。
  • 在包含一万亿时间点的TimeBench上训练,点预测与概率预测均达新高。
  • 适合需要高可靠性、快速生成预测的工业级时间序列应用。

我们提出Sundial,一套原生、灵活且可扩展的时间序列基础模型家族。为预测下一时间片段的分布,我们设计基于流匹配的TimeFlow损失,使Transformer能直接在连续值时间序列上进行原生预训练,无需离散化处理。模型可接受任意长度的时间序列作为条件,预训练时不指定先验分布,能生成多个可能的预测结果,相比参数化密度具有更强表示学习灵活性。为构建时间序列基础模型,我们对Transformer进行最小但关键的改进,并构建了包含一万亿时间点的TimeBench数据集,主要由真实世界数据和合成数据组成。通过TimeFlow损失缓解模式崩溃,我们在TimeBench上预训练了多款Sundial模型,实现了前所未有的模型容量与泛化性能。除卓越可扩展性外,Sundial在点预测与概率预测基准上均达到当前最优表现,且具备即时推理速度——零样本预测仅需数毫秒。我们认为Sundial开创性的生成式预测能力可提升真实决策场景中的模型可靠性。代码已开源:https://github.com/thuml/Sundial。

原文摘要 · Abstract (English)

We introduce Sundial, a family of native, flexible, and scalable time series foundation models. To predict the next-patch's distribution, we propose a TimeFlow Loss based on flow-matching, which facilitates native pre-training of Transformers on continuous-valued time series without discrete tokenization. Conditioned on arbitrary-length time series, our models are pre-trained without specifying any prior distribution and can generate multiple probable predictions, achieving more flexibility in representation learning than using parametric densities. Towards time series foundation models, we leverage minimal but crucial adaptations of Transformers and curate TimeBench with one trillion time points, comprising mostly real-world datasets and synthetic data. By mitigating mode collapse via TimeFlow Loss, we pre-train a family of Sundial models on TimeBench, which achieve unprecedented model capacity and generalization performance. In addition to excellent scalability, Sundial achieves state-of-the-art results on both point and probabilistic forecasting benchmarks with a just-in-time inference speed, i.e., making zero-shot predictions within a few milliseconds. We believe that Sundial's pioneering generative forecasting capability can improve model reliability in real-world decision-making. Code is available at: https://github.com/thuml/Sundial.

时间序列生成模型基础模型流匹配

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。