arXiv:2508.01508cs.LG2025-08

通过频谱初始化与约束优化,让模型更好捕捉长期时间序列中的周期性规律。

Frequency-Constrained Learning for Long-Term Forecasting

  • 用FFT提取低频主成分,初始化正弦嵌入以引入周期先验
  • 采用双速学习率,在训练中保持关键频率结构不被破坏
  • 适用于各类Transformer架构,尤其在长时预测中表现显著

许多现实世界的时间序列受物理规律、人类习惯或季节周期影响,具有强烈的周期性特征。然而,现代深度预测模型常因频谱偏差和缺乏频率感知的归纳偏置而难以捕捉这些重复模式。为此,我们提出一种简单但有效的方法:通过频谱初始化和频率约束优化来增强长期预测能力。具体地,利用快速傅里叶变换(FFT)引导的坐标下降法提取主导低频成分,以此初始化正弦嵌入,并采用双速学习率调度策略,在训练过程中保留有意义的频率结构。该方法具备模型无关性,可无缝集成至现有基于Transformer的架构中。在多个真实世界基准上的大量实验表明,其性能持续提升,尤其是在长预测时域下效果显著,验证了向深度时序模型注入频谱先验对实现鲁棒、可解释的长程预测的价值。此外,在合成数据上,本方法能准确恢复真实频率,进一步证明其在捕捉潜在周期模式方面的有效性与可解释性。

原文摘要 · Abstract (English)

Many real-world time series exhibit strong periodic structures arising from physical laws, human routines, or seasonal cycles. However, modern deep forecasting models often fail to capture these recurring patterns due to spectral bias and a lack of frequency-aware inductive priors. Motivated by this gap, we propose a simple yet effective method that enhances long-term forecasting by explicitly modeling periodicity through spectral initialization and frequency-constrained optimization. Specifically, we extract dominant low-frequency components via Fast Fourier Transform (FFT)-guided coordinate descent, initialize sinusoidal embeddings with these components, and employ a two-speed learning schedule to preserve meaningful frequency structure during training. Our approach is model-agnostic and integrates seamlessly into existing Transformer-based architectures. Extensive experiments across diverse real-world benchmarks demonstrate consistent performance gains--particularly at long horizons--highlighting the benefits of injecting spectral priors into deep temporal models for robust and interpretable long-range forecasting. Moreover, on synthetic data, our method accurately recovers ground-truth frequencies, further validating its interpretability and effectiveness in capturing latent periodic patterns.

时间序列周期性建模Transformer频谱先验

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。