发现下一代储备池计算的数值不稳定性根源，无需正则化也能高效预测混沌序列。
On the emergence of numerical instabilities in Next Generation Reservoir Computing
- 通过数值线性代数与动力系统理论结合，分析特征矩阵条件数变化规律。
- 短时间延迟、高次多项式和数据量少时特征矩阵易病态，导致训练失败。
- SVD算法无需正则化即可实现精准预测，适合追求效率的时序建模任务。
下一代储备池计算（NGRC）是一种低成本机器学习方法，用于从数据中预测混沌时间序列。计算效率对可扩展的储备池计算至关重要，需更优策略降低训练成本。本文揭示了NGRC特征矩阵（由时间延迟坐标上的多项式评估构成）的数值条件性与长期动态之间的关联。我们证明，NGRC可在无正则化情况下训练，从而减少计算时间。贡献有二：其一，融合数值线性代数与动力系统遍历理论，系统研究特征矩阵条件数随超参数的变化规律；结果显示，在短时间延迟、高阶多项式及较短训练数据长度下，特征矩阵趋于病态。其二，评估不同数值算法（Cholesky、奇异值分解SVD、LU分解）求解正则化最小二乘问题的效果；结果表明，基于SVD的训练无需正则化即可实现高精度预测，优于其他算法。
原文摘要 · Abstract (English)
Next Generation Reservoir Computing (NGRC) is a low-cost machine learning method for forecasting chaotic time series from data. Computational efficiency is crucial for scalable reservoir computing, requiring better strategies to reduce training cost. In this work, we uncover a connection between the numerical conditioning of the NGRC feature matrix -- formed by polynomial evaluations on time-delay coordinates -- and the long-term NGRC dynamics. We show that NGRC can be trained without regularization, reducing computational time. Our contributions are twofold. First, merging tools from numerical linear algebra and ergodic theory of dynamical systems, we systematically study how the feature matrix conditioning varies across hyperparameters. We demonstrate that the NGRC feature matrix tends to be ill-conditioned for short time lags, high-degree polynomials, and short length of training data. Second, we evaluate the impact of different numerical algorithms (Cholesky, singular value decomposition (SVD), and lower-upper (LU) decomposition) for solving the regularized least-squares problem. Our results reveal that SVD-based training achieves accurate forecasts without regularization, being preferable when compared against the other algorithms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。