arXiv:2409.17703cs.LG2024-09NeurIPS被引 16

提出新型并行网络PGN,高效捕捉长序列依赖,超越RNN。

PGN: The RNN's New Successor is Effective for Long-Range Time Series Forecasting

  • 设计历史信息提取层与门控机制,实现常数级信息传播路径
  • TPGN在5个基准数据集上达当前最优性能,理论复杂度为O(√L)
  • 适合需要高效处理长序列时间预测的工业场景

由于RNN的循环结构导致长距离信息传播路径受限、梯度爆炸/消失问题以及串行执行效率低下,本文提出一种新范式——并行门控网络(PGN),作为RNN的新替代。PGN通过设计的历史信息提取(HIE)层直接获取过去时间步的信息,并利用门控机制将其与当前时刻信息融合,将信息传播路径压缩至$/mathcal{O}(1)$,有效克服了RNN的局限性。为进一步提升在长序列时间预测任务中的表现,提出新型时序建模框架Temporal PGN(TPGN),包含两条分支:一条使用PGN捕捉长期周期模式并保留局部特征;另一条采用分块策略捕获短期信息并聚合全局表示。TPGN实现理论复杂度$/mathcal{O}( ext{√}L)$,确保运算高效。在五个基准数据集上的实验结果表明,TPGN达到当前最优(SOTA)性能且运行高效,进一步验证了PGN作为RNN新替代者的有效性。代码已开源。

原文摘要 · Abstract (English)

Due to the recurrent structure of RNN, the long information propagation path poses limitations in capturing long-term dependencies, gradient explosion/vanishing issues, and inefficient sequential execution. Based on this, we propose a novel paradigm called Parallel Gated Network (PGN) as the new successor to RNN. PGN directly captures information from previous time steps through the designed Historical Information Extraction (HIE) layer and leverages gated mechanisms to select and fuse it with the current time step information. This reduces the information propagation path to $\mathcal{O}(1)$, effectively addressing the limitations of RNN. To enhance PGN's performance in long-range time series forecasting tasks, we propose a novel temporal modeling framework called Temporal PGN (TPGN). TPGN incorporates two branches to comprehensively capture the semantic information of time series. One branch utilizes PGN to capture long-term periodic patterns while preserving their local characteristics. The other branch employs patches to capture short-term information and aggregate the global representation of the series. TPGN achieves a theoretical complexity of $\mathcal{O}(\sqrt{L})$, ensuring efficiency in its operations. Experimental results on five benchmark datasets demonstrate the state-of-the-art (SOTA) performance and high efficiency of TPGN, further confirming the effectiveness of PGN as the new successor to RNN in long-range time series forecasting. The code is available in this repository: \url{https://github.com/Water2sea/TPGN}.

时间序列并行计算长程依赖PGN

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。