提出神经振荡器的理论逼近上限,证明其高效建模动态系统。
Upper Approximation Bounds for Neural Oscillators
- 基于二阶微分方程与MLP构建神经振荡器,推导其逼近能力上界。
- 逼近误差随MLP宽度倒数呈多项式下降,突破参数复杂性瓶颈。
- 适用于物理系统建模与时序因果学习,适合科学工程领域研究者。
神经振荡器源自二阶常微分方程,已在长期序列或连续时间函数间的因果映射稳定学习及物理系统精确逼近中表现出色。然而,其神经网络架构的理论容量量化仍是重大挑战。本文研究由二阶微分方程后接多层感知机(MLP)构成的神经振荡器,推导其在连续时间函数空间间近似因果连续算子,以及近似一致渐近增量稳定的二阶动力系统时的上界逼近误差。所建立的证明方法可直接推广至线性时连续复杂循环神经网络加MLP构成的状态空间模型。理论结果表明,神经振荡器对二阶动力系统的逼近误差随两个使用MLP宽度的倒数呈多项式衰减,从而克服参数复杂性困境。通过四个数值实验验证了两个逼近误差上界的收敛速率。这些结果为神经振荡器在科学与工程中的有效应用提供了坚实的理论基础。
原文摘要 · Abstract (English)
Neural oscillators, originating from second-order ordinary differential equations (ODEs), have demonstrated strong performance in stably learning causal mappings between long-term sequences or continuous temporal functions, as well as in accurately approximating physical systems. However, theoretically quantifying the capacities of their neural network architectures remains a significant challenge. In this study, the neural oscillator consisting of a second-order ODE followed by a multilayer perceptron (MLP) is considered. Its upper approximation bound for approximating causal and uniformly continuous operators between continuous temporal function spaces and that for approximating uniformly asymptotically incrementally stable second-order dynamical systems are derived. The established proof method of the approximation bound for approximating the causal continuous operators can also be directly applied to state-space models consisting of a linear time-continuous complex recurrent neural network followed by an MLP. Theoretical results reveal that the approximation error of the neural oscillator for approximating the second-order dynamical systems scales polynomially with the reciprocals of the widths of two utilized MLPs, thus overcoming the curse of parametric complexity. The convergence rates of two established approximation error bounds are validated through four numerical cases. These results provide a robust theoretical foundation for the effective application of the neural oscillator in science and engineering.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。