PatchFormer通过分块自监督预训练，实现少数据零样本多时序预测。
PatchFormer: A Patch-Based Time Series Foundation Model with Hierarchical Masked Reconstruction and Cross-Domain Transfer Learning for Zero-Shot Multi-Horizon Forecasting
- 将时间序列分块，用层级掩码重建学习多尺度特征
- 零样本预测误差比基线低27.3%，任务训练数据只需6%
- 适合缺乏标注数据的跨领域时序预测场景
时间序列预测在气候、能源、医疗和金融等领域具有基础性应用。现有方法通常依赖特定领域特征工程和大量标注数据。我们提出PatchFormer，一种基于分块的时间序列基础模型，采用层级掩码重建进行自监督预训练，并使用轻量适配器实现高效迁移。该模型将时间序列划分为若干分块，通过可学习聚合机制在多时间尺度上学习表征。预训练阶段采用动态掩码的分块重建策略，同时优化局部精度与全局一致性；随后引入跨域知识蒸馏。在涵盖天气、能源、交通、金融和医疗的24个基准数据集上的实验表明，其在零样本多步预测中达到领先性能，相比强基线平均均方误差降低27.3%，且任务微调所需数据仅需6%（即减少94%）。模型在超过1000亿个时间点的预训练数据下表现出接近对数线性的扩展能力，处理长度为512的序列速度比全序列变换器快3.8倍。
原文摘要 · Abstract (English)
Time series forecasting is a fundamental problem with applications in climate, energy, healthcare, and finance. Many existing approaches require domain-specific feature engineering and substantial labeled data for each task. We introduce PatchFormer, a patch-based time series foundation model that uses hierarchical masked reconstruction for self-supervised pretraining and lightweight adapters for efficient transfer. PatchFormer segments time series into patches and learns multiscale temporal representations with learnable aggregation across temporal scales. Pretraining uses masked patch reconstruction with dynamic masking and objectives that encourage both local accuracy and global consistency, followed by cross-domain knowledge distillation. Experiments on 24 benchmark datasets spanning weather, energy, traffic, finance, and healthcare demonstrate state-of-the-art zero-shot multi-horizon forecasting, reducing mean squared error by 27.3 percent relative to strong baselines while requiring 94 percent less task-specific training data. The model exhibits near log-linear scaling with more pretraining data up to 100 billion points and processes length-512 sequences 3.8x faster than full-sequence transformers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。