arXiv:2503.15696math.NAcs.LG2025-03被引 4

证明了神经微分方程可逼近任意连续函数,且约束后仍保持良好逼近能力。

Approximation properties of neural ODEs

  • 将神经ODE与嵌入/投影结合,构建特殊激活函数的浅层网络
  • 在独立约束流映射的Lipschitz常数或权重范数时,仍具通用逼近性
  • 同时施加两类约束会降低表达力,但给出可量化的逼近误差上界

我们研究神经常微分方程(neural ODE)在连续函数空间中的逼近性质。由于neural ODE要求输入输出维度相同,而一般连续函数的输入输出维数不同,需将输入嵌入到neural ODE的隐空间,并将输出从隐空间投影回目标空间。通过组合neural ODE的流映射与嵌入、投影操作,得到一个浅层神经网络,其激活函数为neural ODE在积分区间终点的流映射。因此,neural ODE的逼近性质等价于该类浅层网络的逼近性质。我们证明了此类浅层网络在连续函数空间中具有通用逼近性(UAP)。进一步,研究了参数受特定约束时的逼近性能:限制neural ODE流映射的Lipschitz常数和权重范数以增强网络稳定性。证明当两类约束单独作用时,仍保持UAP;当两者同时施加时,表达力下降,我们推导出量化逼近精度的误差上界。

原文摘要 · Abstract (English)

We study the approximation properties of neural ordinary differential equations (neural ODEs) in the space of continuous functions. Since a neural ODE requires input and output dimensions to be the same, while input and output dimensions of a continuous function are generally different, we need to embed an input into the latent space of the neural ODE, and to project the output of the neural ODE into the output space. By composing the neural ODE flow map with such embedding and projection operations, we get a shallow neural network whose activation function is defined as the flow map of the neural ODE at the final time of the integration interval. Thus, the study of the approximation properties of neural ODEs leads to the study of the approximation properties of shallow neural networks with a particular choice of activation function. We prove the universal approximation property (UAP) of such shallow neural networks in the space of continuous functions. Furthermore, we investigate the approximation properties of shallow neural networks whose parameters satisfy specific constraints. In particular, we constrain the Lipschitz constant of the neural ODE's flow map and the norms of the weights to increase the network's stability. We prove that the UAP holds if we consider either constraint independently. When both are enforced, there is a loss of expressiveness, and we derive approximation bounds that quantify how accurately such a constrained network can approximate a continuous function.

神经ODE逼近理论通用逼近性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。