arXiv:2503.04068math.OCcs.LG2025-03

窄神经微分方程能高效逼近宽网络的动态流,且证明更简洁。

Quantitative Flow Approximation Properties of Narrow Neural ODEs

  • 用常微分方程和Gröwnall引理给出新证明方法
  • 指出窄网络需约10次权重切换即可模拟宽网络行为
  • 适合研究神经微分方程理论与控制的读者

本文重新审视神经微分方程(NODEs)的流逼近性质。当神经ODE的参数维度等于输入维度时,称为‘窄’神经ODE,其宽度受限。我们推导了窄NODE逼近浅层但宽节点的流之间的关系。基于浅层神经网络的逼近结果,这有助于理解哪些动力系统流可由窄神经ODE逼近。尽管已有文献建立窄NODE的逼近性,但证明通常涉及复杂构造或依赖控制论中的深奥可控性定理。本文提供一种仅基于常微分方程与Gröwnall引理的简化证明。此外,我们给出了窄神经ODE时间依赖权重所需切换次数的估计,使其能模仿单层宽神经网络作为速度场的行为。

原文摘要 · Abstract (English)

In this note, we revisit the problem of flow approximation properties of neural ordinary differential equations (NODEs). The approximation properties have been considered as a flow controllability problem in recent literature. The neural ODE is considered {\it narrow} when the parameters have dimension equal to the input of the neural network, and hence have limited width. We derive the relation of narrow NODEs in approximating flows of shallow but wide NODEs. Due to existing results on approximation properties of shallow neural networks, this facilitates understanding which kind of flows of dynamical systems can be approximated using narrow neural ODEs. While approximation properties of narrow NODEs have been established in literature, the proofs often involve extensive constructions or require invoking deep controllability theorems from control theory. In this paper, we provide a simpler proof technique that involves only ideas from ODEs and Gr{ö}nwall's lemma. Moreover, we provide an estimate on the number of switches needed for the time dependent weights of the narrow NODE to mimic the behavior of a NODE with a single layer wide neural network as the velocity field.

神经ODE流逼近数学分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。