arXiv:2509.24122cs.LG2025-09

用高效神经网络捕捉长期时间依赖,预测更准且训练更快。

Echo Flow Networks

  • 多分支回声网络结合新型门控激活机制,提升表达能力。
  • 训练速度提升4倍,模型体积缩小3倍,误差降低20%。
  • 适合需要长序列建模的时序预测任务,如气象与交通预测。

时间序列预测的核心挑战在于如何高效捕捉超长序列中的长期依赖关系。尽管深度学习取得进展,传统架构常在计算复杂度与信息保留之间权衡。回声状态网络(ESNs)因其恒定内存和每步训练复杂度,近年来备受关注,适用于极长历史建模。但传统ESNs受限于非线性能力,影响表现与稳定性。本文提出回声流网络(EFNs),由多个扩展型回声状态网络(X-ESNs)与MLP读出层组成,引入新颖的矩阵门控复合随机激活(MCRA),实现神经元级复杂时序动态,显著提升表示能力而不牺牲效率。此外,设计双流结构,近期输入动态选择无限视野记忆中的特征,增强预测精度与长期稳定性。在五个基准测试上,EFNs相比PatchTST实现最高4倍训练加速、3倍模型压缩,预测误差从43%降至35%,相对改进20%。其中一实例EchoFormer在ETTh、ETTm、DMV、Weather和Air Quality数据集上均达到新最优性能。

原文摘要 · Abstract (English)

At the heart of time-series forecasting (TSF) lies a fundamental challenge: how can models efficiently and effectively capture long-range temporal dependencies across ever-growing sequences? While deep learning has brought notable progress, conventional architectures often face a trade-off between computational complexity and their ability to retain accumulative information over extended horizons. Echo State Networks (ESNs), a class of reservoir computing models, have recently regained attention for their exceptional efficiency, offering constant memory usage and per-step training complexity regardless of input length. This makes them particularly attractive for modeling extremely long-term event history in TSF. However, traditional ESNs fall short of state-of-the-art performance due to their limited nonlinear capacity, which constrains both their expressiveness and stability. We introduce Echo Flow Networks (EFNs), a framework composed of a group of extended Echo State Networks (X-ESNs) with MLP readouts, enhanced by our novel Matrix-Gated Composite Random Activation (MCRA), which enables complex, neuron-specific temporal dynamics, significantly expanding the network's representational capacity without compromising computational efficiency. In addition, we propose a dual-stream architecture in which recent input history dynamically selects signature reservoir features from an infinite-horizon memory, leading to improved prediction accuracy and long-term stability. Extensive evaluations on five benchmarks demonstrate that EFNs achieve up to 4x faster training and 3x smaller model size compared to leading methods like PatchTST, reducing forecasting error from 43% to 35%, a 20% relative improvement. One instantiation of our framework, EchoFormer, consistently achieves new state-of-the-art performance across five benchmark datasets: ETTh, ETTm, DMV, Weather, and Air Quality.

时间序列回声网络高效建模预测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。