动态调整长短时依赖权重，提升时间序列预测精度与效率
ParallelTime: Dynamically Weighting the Balance of Short- and Long-Term Temporal Dependencies
- 提出并行加权机制，根据输入动态分配长短时依赖权重
- 在多个基准上达到领先性能，参数更少、计算量更低
- 适合需要高效长时预测的工业级时间序列场景
现代多变量时间序列预测主要依赖Transformer和Mamba两种架构。在自然语言处理中，常将局部窗口注意力（用于捕捉短时依赖）与Mamba（用于捕捉长时依赖）的输出平均，赋予两者相等权重。我们发现，对于时间序列预测任务，这种等权重分配并非最优。为此，我们提出动态加权机制ParallelTime Weighter，基于输入和模型知识为每个令牌计算长短时依赖的相互关联权重。进一步，我们构建ParallelTime架构，集成该加权机制，在多个基准上实现领先性能。该架构具备更强鲁棒性，显著降低FLOPs，参数更少，可有效扩展至更长预测范围，并大幅超越现有方法。这些进展为未来并行注意力-Mamba架构在时间序列预测中的发展指明了方向。代码已开源：https://github.com/itay1551/ParallelTime。
原文摘要 · Abstract (English)
Modern multivariate time series forecasting primarily relies on two architectures: the Transformer with attention mechanism and Mamba. In natural language processing, an approach has been used that combines local window attention for capturing short-term dependencies and Mamba for capturing long-term dependencies, with their outputs averaged to assign equal weight to both. We find that for time-series forecasting tasks, assigning equal weight to long-term and short-term dependencies is not optimal. To mitigate this, we propose a dynamic weighting mechanism, ParallelTime Weighter, which calculates interdependent weights for long-term and short-term dependencies for each token based on the input and the model's knowledge. Furthermore, we introduce the ParallelTime architecture, which incorporates the ParallelTime Weighter mechanism to deliver state-of-the-art performance across diverse benchmarks. Our architecture demonstrates robustness, achieves lower FLOPs, requires fewer parameters, scales effectively to longer prediction horizons, and significantly outperforms existing methods. These advances highlight a promising path for future developments of parallel Attention-Mamba in time series forecasting. The implementation is readily available at: \href{https://github.com/itay1551/ParallelTime}{GitHub}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。