用合成数据训练线性RNN，实现高效零样本时间序列预测
TempoPFN: Synthetic Pre-training of Linear RNNs for Zero-shot Time Series Forecasting
- 基于线性RNN与状态编织结构，支持全并行训练推理
- 在多个基准上超越多数真实数据训练模型，且效率更高
- 适合需要可复现、高效零样本预测的研究者使用
面向零样本时间序列预测的基座模型面临长程预测效率低和可复现性差的问题，现有纯合成数据方法在挑战性基准上表现不佳。本文提出TempoPFN，一种基于线性循环神经网络的单变量时间序列基座模型，仅在合成数据上预训练。模型采用GatedDeltaProduct架构与状态编织机制，实现跨序列长度的完全并行训练，无需窗口化或摘要技术，同时保持稳定的时序状态追踪。我们构建了统一的合成数据流水线，整合随机微分方程、高斯过程与音频合成等多种生成器，并引入新颖增强策略。在Gift-Eval、fev-bench和Chronos-ZS基准上的零样本评估中，TempoPFN达到顶级性能，超越所有现有纯合成数据方法，并优于多数真实数据训练模型；同时相比现有基线更高效，得益于完全并行的训练与推理。我们开源了完整的数据生成流水线与训练代码，为未来研究提供可复现基础。
原文摘要 · Abstract (English)
Foundation models for zero-shot time series forecasting face challenges in efficient long-horizon prediction and reproducibility, with existing synthetic-only approaches underperforming on challenging benchmarks. This paper presents TempoPFN, a univariate time series foundation model based on linear Recurrent Neural Networks (RNNs) pre-trained exclusively on synthetic data. The model uses a GatedDeltaProduct architecture with state-weaving for fully parallelizable training across sequence lengths, eliminating the need for windowing or summarization techniques while maintaining robust temporal state-tracking. Our comprehensive synthetic data pipeline unifies diverse generators, including stochastic differential equations, Gaussian processes, and audio synthesis, with novel augmentations. In zero-shot evaluations on the Gift-Eval, fev-bench and Chronos-ZS benchmarks, TempoPFN achieves top-tier competitive performance, outperforming all existing synthetic-only approaches and surpassing the majority of models trained on real-world data, while being more efficient than existing baselines by leveraging fully parallelizable training and inference. We open-source our complete data generation pipeline and training code, providing a reproducible foundation for future research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。