提出频谱注意力机制，让模型轻松捕捉数千步的长期依赖。
Introducing Spectral Attention for Long-Range Dependency in Time Series Forecasting
- 用频谱注意力保留样本间时间相关性，避免打乱顺序导致信息丢失。
- 在11个真实数据集上，7种模型均显著提升预测精度，达到新纪录。
- 可无缝嵌入主流序列模型，无需改动基础结构，适合长期时序预测场景。
序列建模在捕捉跨任务的长程依赖方面面临挑战。近年来，基于线性与Transformer的预测模型在时间序列预测中表现优异，但受限于固定大小输入进行预测，难以有效处理长程依赖。此外，通常通过打乱样本组成小批量训练，牺牲了连续样本间的必要时间相关性。为克服这些限制，本文提出一种快速高效的频谱注意力机制，能保持样本间的时间相关性，促进梯度在样本间流动，并支持长程信息处理，同时不改变基础模型结构。该机制通过低通滤波保留长期趋势，帮助梯度传播。频谱注意力可无缝集成到多数序列模型中，使具有固定回看窗口的模型能够捕捉数千步的长程依赖。在11个真实世界时间序列数据集上，使用7种近期预测模型进行的大量实验表明，该方法持续有效，实现当前最优性能。
原文摘要 · Abstract (English)
Sequence modeling faces challenges in capturing long-range dependencies across diverse tasks. Recent linear and transformer-based forecasters have shown superior performance in time series forecasting. However, they are constrained by their inherent inability to effectively address long-range dependencies in time series data, primarily due to using fixed-size inputs for prediction. Furthermore, they typically sacrifice essential temporal correlation among consecutive training samples by shuffling them into mini-batches. To overcome these limitations, we introduce a fast and effective Spectral Attention mechanism, which preserves temporal correlations among samples and facilitates the handling of long-range information while maintaining the base model structure. Spectral Attention preserves long-period trends through a low-pass filter and facilitates gradient to flow between samples. Spectral Attention can be seamlessly integrated into most sequence models, allowing models with fixed-sized look-back windows to capture long-range dependencies over thousands of steps. Through extensive experiments on 11 real-world time series datasets using 7 recent forecasting models, we consistently demonstrate the efficacy of our Spectral Attention mechanism, achieving state-of-the-art results.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。