arXiv:2603.04791cs.AI2026-03被引 14

超大规模时间序列模型,长序列预测更准且不累积误差

Timer-S1: A Billion-Scale Time Series Foundation Model with Serial Scaling

  • 用串行预测机制替代传统逐步预测,避免误差累积
  • 83亿参数,每令牌激活7.5亿,支持1.15万长度上下文
  • 自建万亿时间点数据集,适合长期预测研究者使用

我们提出Timer-S1,一个具有83亿总参数、每令牌激活7.5亿参数、上下文长度达1.15万的强大学习型时间序列基础模型。为突破现有预训练时间序列模型的可扩展性瓶颈,我们在模型架构、数据集和训练流程三个维度实施串行缩放(Serial Scaling)。Timer-S1融合稀疏TimeMoE块与通用TimeSTP块,采用遵循时序特性的串行令牌预测(STP)目标函数,通过串行计算提升长期预测能力,同时避免昂贵的滚动推理和显著的误差累积问题。为获取高质量无偏数据集,我们构建包含一万亿时间点的TimeBench语料库,并进行精细的数据增强以缓解预测偏差。此外,首创后训练阶段,包括持续预训练与长上下文扩展,进一步提升短中长期性能。在GIFT-Eval大规模评测基准上,Timer-S1作为预训练模型取得最优的MASE和CRPS得分,表现领先。模型已开源,以推动后续研究。

原文摘要 · Abstract (English)

We introduce Timer-S1, a strong Mixture-of-Experts (MoE) time series foundation model with 8.3B total parameters, 0.75B activated parameters for each token, and a context length of 11.5K. To overcome the scalability bottleneck in existing pre-trained time series foundation models, we perform Serial Scaling in three dimensions: model architecture, dataset, and training pipeline. Timer-S1 integrates sparse TimeMoE blocks and generic TimeSTP blocks for Serial-Token Prediction (STP), a generic training objective that adheres to the serial nature of forecasting. The proposed paradigm introduces serial computations to improve long-term predictions while avoiding costly rolling-style inference and pronounced error accumulation in the standard next-token prediction. Pursuing a high-quality and unbiased training dataset, we curate TimeBench, a corpus with one trillion time points, and apply meticulous data augmentation to mitigate predictive bias. We further pioneer a post-training stage, including continued pre-training and long-context extension, to enhance short-term and long-context performance. Evaluated on the large-scale GIFT-Eval leaderboard, Timer-S1 achieves state-of-the-art forecasting performance, attaining the best MASE and CRPS scores as a pre-trained model. Timer-S1 is released to facilitate further research.

时间序列大模型预测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。