让递归神经网络并行处理时序数据，提升效率与扩展性
ParalESN: Enabling parallel information processing in Reservoir Computing
- 采用复数域对角线递推机制，实现时序数据并行计算
- 在保持传统模型性能前提下，计算开销降低多个数量级
- 适合需要高效时序建模的深度学习场景
递归计算（Reservoir Computing, RC）已成为高效处理时序数据的范式。然而，其可扩展性受限于序列化处理需求及高维蓄水池带来的巨大内存开销。本文从结构化算子与状态空间建模视角重新审视RC，提出并行回声状态网络（ParalESN）。通过复数域对角线线性递推，ParalESN 实现时序数据的并行处理，并构建高效高维蓄水池。理论分析表明，传统回声状态网络的回声状态性质与通用性保证得以保留，且任意线性蓄水池均可等价表示为复数对角形式。实验显示，ParalESN 在预测精度上与传统RC及全可训练序列模型相当，同时实现数量级的计算节省。整体而言，ParalESN 为将RC融入深度学习体系提供了可扩展、有原则的路径。
原文摘要 · Abstract (English)
Reservoir Computing (RC) has established itself as an efficient paradigm for temporal processing. However, its scalability remains severely constrained by the need to process temporal data sequentially and the prohibitive memory footprint of high-dimensional reservoirs. To address these limitations, we revisit RC through the lens of structured operators and state space modeling, introducing Parallel Echo State Network (ParalESN). Leveraging diagonal linear recurrence in the complex domain, ParalESN enables parallel processing of temporal data and the construction of efficient, high-dimensional reservoirs. A thorough theoretical analysis demonstrates that the Echo State Property and the universality guarantees of traditional Echo State Networks are preserved, while also admitting an equivalent representation of arbitrary linear reservoirs in the complex diagonal form. Empirically, ParalESN achieves competitive predictive accuracy with traditional RC and with fully trainable sequence models, while delivering computational savings by orders of magnitude. Overall, ParalESN offers a scalable and principled pathway for integrating RC within the deep learning landscape.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。