用随机矩阵理论解析了时延神经网络的误差特性，给出最优正则化方法。
A Random Matrix Perspective of Echo State Networks: From Precise Bias--Variance Characterization to Optimal Regularization
- 基于随机矩阵理论推导出误差的闭式表达式
- 发现时延网络无双下降现象，小样本下表现更优
- 提供可计算的最优正则化方案，适合工程调参
我们在线性教师-学生设定下对时延神经网络（ESN）进行严格的渐近分析，假设教师权重已知。借助随机矩阵理论，我们推导出渐近偏差、方差和均方误差（MSE）关于输入统计量、真实权重向量及岭正则化参数的闭式表达式。分析揭示两个关键差异：（i）ESN 不呈现双下降现象；（ii）当训练样本数和教师记忆长度均受限时，ESN 能达到更低的 MSE。我们进一步给出了身份输入协方差情形下的最优正则化显式公式，并提出一种高效数值算法以计算一般情况下的最优值。这些结果为 ESN 提供了可解释的理论基础与实用调参指导，有助于将近期经验观察与可证明的性能保证相统一。
原文摘要 · Abstract (English)
We present a rigorous asymptotic analysis of Echo State Networks (ESNs) in a teacher student setting with a linear teacher with oracle weights. Leveraging random matrix theory, we derive closed form expressions for the asymptotic bias, variance, and mean-squared error (MSE) as functions of the input statistics, the oracle vector, and the ridge regularization parameter. The analysis reveals two key departures from classical ridge regression: (i) ESNs do not exhibit double descent, and (ii) ESNs attain lower MSE when both the number of training samples and the teacher memory length are limited. We further provide an explicit formula for the optimal regularization in the identity input covariance case, and propose an efficient numerical scheme to compute the optimum in the general case. Together, these results offer interpretable theory and practical guidelines for tuning ESNs, helping reconcile recent empirical observations with provable performance guarantees
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。