将递归网络重新建模为状态空间模型,揭示其稳定性和记忆机制。
Echo State Networks as State-Space Models: A Systems Perspective
- 从系统理论出发,将回声状态网络视为带收缩特性的非线性状态空间模型。
- 推导出确保系统稳定的可验证条件,涵盖泄漏率、谱缩放和激活函数的Lipschitz常数。
- 提出基于卡尔曼滤波的读出训练方法,支持超参数联合优化与谱形塑造。
回声状态网络(ESNs)通常被视为高效的、仅读出层可训练的递归模型,但其动态行为和设计多依赖启发式而非第一性原理。本文将其明确重构为状态空间模型(SSMs),建立水库计算与经典系统辨识及现代核化状态空间模型之间的统一系统理论框架。首先,证明回声状态性质是满足压缩非线性状态空间模型输入到状态稳定性的一种情形,并推导出关于泄漏率、谱缩放和激活函数Lipschitz常数的可验证条件。其次,提出两种互补映射:(i) 小信号线性化得到局部有效的线性时不变(LTI)状态空间模型,具备可解释的极点和记忆时长;(ii) 提升/科普曼随机特征展开,使ESN在扩展状态空间中表现为线性状态空间模型,从而支持传递函数与卷积核分析。该视角实现了对记忆谱的频域表征,并阐明了何时ESN可模拟结构化的状态空间核。第三,将教师强迫视为状态估计问题,提出基于卡尔曼/扩展卡尔曼滤波的读出学习方法,结合期望最大化算法(EM)联合优化超参数(泄漏率、谱半径、过程/测量噪声),并设计一种混合子空间方法,在收缩约束下实现谱形调控。
原文摘要 · Abstract (English)
Echo State Networks (ESNs) are typically presented as efficient, readout-trained recurrent models, yet their dynamics and design are often guided by heuristics rather than first principles. We recast ESNs explicitly as state-space models (SSMs), providing a unified systems-theoretic account that links reservoir computing with classical identification and modern kernelized SSMs. First, we show that the echo-state property is an instance of input-to-state stability for a contractive nonlinear SSM and derive verifiable conditions in terms of leak, spectral scaling, and activation Lipschitz constants. Second, we develop two complementary mappings: (i) small-signal linearizations that yield locally valid LTI SSMs with interpretable poles and memory horizons; and (ii) lifted/Koopman random-feature expansions that render the ESN a linear SSM in an augmented state, enabling transfer-function and convolutional-kernel analyses. This perspective yields frequency-domain characterizations of memory spectra and clarifies when ESNs emulate structured SSM kernels. Third, we cast teacher forcing as state estimation and propose Kalman/EKF-assisted readout learning, together with EM for hyperparameters (leak, spectral radius, process/measurement noise) and a hybrid subspace procedure for spectral shaping under contraction constraints.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。