提出可感知时间尺度的预训练方法，提升多频率时序模型性能与效率。
Scale-Aware Pretraining of Time Series Foundation Models via Multi-Patch Token Alignment and Hybrid Masking
- 引入尺度感知的令牌对齐机制，动态适应不同采样频率。
- 在LSTF基准上实现MSE降低9.2%、MASE提升8.3%，效率提高65.6%。
- 适合需要跨频段时序建模的工业预测与金融分析场景。
在异构数据集上预训练时序基础模型需有效处理不同采样频率。现有方法或采用数据集特异性块大小和独立前馈网络，导致表征碎片化；或强制固定块大小，忽略固有时序差异。为此，本文提出SATP，通过尺度感知的令牌对齐机制，将块大小显式视为尺度概念。结合对比学习启发的对齐正则项，实现跨尺度表征空间对齐，同时保留各自建模能力。进一步设计混合掩码策略，融合随机与连续掩码，以捕捉多尺度时序结构。在LSTF基准上的实验表明，SATP相比竞争基线在MSE上提升9.2%，GIFT-Eval MASE提升8.3%，且模型效率提升65.6%，展现出显著有效性与可扩展性。
原文摘要 · Abstract (English)
Pretraining time series foundation models across heterogeneous datasets necessitates effective handling of varying sampling frequencies. Current methods either employ dataset-specific patch sizes and separate FFNs, leading to fragmented representations, or enforce a fixed patch size that neglects inherent temporal variations. To address this, we propose SATS, featuring a scale-aware token alignment mechanism that treats patch size as an explicit notion of scale. By incorporating a contrastive-inspired alignment regularizer, SATS aligns representation spaces across scales while preserving distinct modeling capacities. Furthermore, a hybrid masking strategy combining random and contiguous masking is introduced to capture multi-scale temporal structures. Experimental results on LSTF benchmarks demonstrate that SATS achieves a 9.2% improvement in MSE and an 8.3% gain in GIFT-Eval MASE compared to competitive baselines. Notably, SATS consistently delivers SOTA performance while achieving a 65.6% increase in model efficiency over advanced baselines, highlighting its effectiveness and scalability in time series pretraining.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。