揭示了循环神经网络吸引子的维度上限,解释其为何能高效模拟混沌系统。
On the dimension of pullback attractors in recurrent neural networks
- 用非自治动力系统方法推导出吸引子维数的上界。
- 输入来自Nin维系统时,吸引子维数不超过Nin。
- 为储层计算成功模拟混沌系统提供了理论支持,适合研究动态系统与神经网络交叉的学者。
通过储层计算范式训练的循环神经网络在学习和重构混沌系统吸引子方面表现出色,常能复现李雅普诺夫指数和分形维数等量。近期有猜想认为,这是因为储层计算机在状态空间中嵌入了混沌系统的动态。该猜想已在激活函数为线性的情况下被证明,但对更一般的储层系统仍待解决。本文采用非自治动力系统方法,建立了拉回吸引子(储层状态空间中训练和预测阶段近似的目标子集)的盒计数维数上界。证明了拉回吸引子的盒计数维数不超过输入序列空间在乘积拓扑下的盒计数维数。特别地,当输入序列源自Nin维光滑动力系统或其连续可微的通用观测时,拉回吸引子的盒计数维数上界为Nin。结果表明,尽管储层计算机的状态空间可能高维,其实际动态却具有有效低维特性。本研究部分解释了储层计算机在吸引子重构及李雅普诺夫指数、分形维数等动态不变量计算任务中的成功原因。
原文摘要 · Abstract (English)
Recurrent neural networks trained via the reservoir computing paradigm have demonstrated remarkable success in learning and reconstructing attractors from chaotic systems, often replicating quantities such as Lyapunov exponents and fractal dimensions. It has recently been conjectured that this is because the reservoir computer embeds the dynamics of the chaotic system in its state space before learning. This conjecture has been established for reservoir computers with linear activation functions and remains open for more general reservoir systems. In this work, we employ a non-autonomous dynamical systems approach to establish an upper bound for the box-counting dimension of the pullback attractor, a subset of the reservoir state space that is approximated during training and prediction phases. We prove that the box-counting dimension of the pullback attractor is bounded above by the box-counting dimension of the space of input sequences with respect to the product topology. In particular, for input sequences originating from an Nin-dimensional smooth dynamical system or their generic continuously differentiable observations, the box-counting dimension of the pullback attractor is bounded above by Nin. The results obtained here highlight the fact that, while a reservoir computer may possess a very high-dimensional state space, it exhibits effective low-dimensional dynamics. Our findings also partly explain why reservoir computers are successful in tasks such as attractor reconstruction and the computation of dynamic invariants like Lyapunov exponents and fractal dimensions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。