基于输入数据设计储层,提升预测精度与训练稳定性。
Data-Specific Hyper-Parameter Design: A Paradigm Shift in Reservoir Computing

- 从几何角度设计储层,使状态增量对齐输入决定的子空间。
- 小锥角下状态方差集中,显著降低岭回归误差。
- 适用于确定性动力系统输入,适合高维时序建模任务。
传统储层计算依赖大规模随机生成的储层,读出层通常为线性结构。过去二十年的设计主要受稳定性条件约束,如状态收缩或记忆容量,但这些方法与输入数据和学习目标无关,导致依赖随机性的试错过程。在高维情况下,储层作为输入历史的随机嵌入,隐含依赖Johnson-Lindenstrauss型集中现象以保持信息。本文从几何视角出发,针对由确定性动力系统生成的输入,提出新型储层设计原则:要求储层状态增量在输入决定的向量子空间周围形成小锥角集中。理论证明,这种锥集中可降低岭回归训练误差。当锥角较小时,状态方差集中在输入决定的子空间内,改善经验二阶矩矩阵的条件数,并增强主协方差方向与状态-目标交叉协方差的对齐度。对于回声状态网络,我们给出可构造的储层设计方法:使关联的Krylov链方向在输入决定的子空间内近似闭合,同时在其正交补空间中允许可控混合。此外,我们提供一种谱诊断工具,识别何时储层几何将预测信息集中于少数主导协方差模式,以及何时存在‘谱污染’阻碍预测。数值实验表明,该方法在性能上持续优于任意随机构造的储层。
原文摘要 · Abstract (English)
Reservoir computing typically relies on large, randomly generated reservoirs, enabling simple, often linear readouts. Over the past two decades, most constructions have exploited the freedom to select the reservoir, constrained primarily by stability conditions based on state contraction or memory capacity. However, these designs are largely independent of the input data and learning objective, resulting in a trial-and-error methodology driven by randomness. In high dimensions, the reservoir acts as a random embedding of the input history, implicitly relying on Johnson--Lindenstrauss--type concentration phenomena to preserve information. In contrast, we develop reservoir design principles from a geometric perspective for inputs generated by deterministic dynamical systems. Rather than relying on random embeddings, we require reservoir state increments to align within a cone around an input-determined vector subspace, and prove that such a cone concentration reduces ridge-regression training error. When the cone angle is small, the variance of reservoir states concentrates in the input-determined subspace, improving conditioning of the empirical second-moment matrix and strengthening alignment between dominant covariance directions and the state-target cross-covariance. For echo state networks, we provide a constructive approach to reservoir design. The reservoir matrix is chosen so that associated Krylov-chain directions remain nearly closed within an input-determined subspace while permitting controlled mixing in its orthogonal complement. We also provide a spectral diagnostic for ridge regression training that identifies when reservoir geometry concentrates predictive information into a few dominant covariance modes and when ``spectral pollution'' inhibits forecasting. Numerical experiments demonstrate consistent performance gains over arbitrary reservoir constructions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。