首次为非线性神经微分方程推导出通用泛化界，揭示过参数化与域约束的影响。
Generalization Bound for a General Class of Neural Ordinary Differential Equations
- 基于状态变量Lipschitz连续性，分析非线性神经ODE解的有界变差性质。
- 在时间依赖和独立情形下均建立泛化误差上界，支持过参数化与域约束分析。
- 适用于连续深度模型理论分析，适合研究泛化性能的从业者。
神经常微分方程（neural ODEs）是一种具有连续深度架构的流行深度学习模型。为评估其在未见数据上的表现，理解其泛化误差界至关重要。以往研究主要集中在神经ODE中动力学函数的线性情形（Marion, P., 2023），或针对神经控制型ODE给出依赖采样间隔的界（Bleistein et al., 2023）。本文分析一类更广泛的神经ODE，其中动力学函数为一般非线性函数，可为时间相关或无关，且关于状态变量满足Lipschitz连续性。我们证明在此Lipschitz条件下，神经ODE的解具有有界变差。基于此，我们推导了时间依赖与时间独立情形下的泛化界，并研究了过参数化与域约束对这些界的影响。据我们所知，这是首个针对具一般非线性动力学的神经ODE的泛化界推导。
原文摘要 · Abstract (English)
Neural ordinary differential equations (neural ODEs) are a popular type of deep learning model that operate with continuous-depth architectures. To assess how well such models perform on unseen data, it is crucial to understand their generalization error bounds. Previous research primarily focused on the linear case for the dynamics function in neural ODEs - Marion, P. (2023), or provided bounds for Neural Controlled ODEs that depend on the sampling interval Bleistein et al. (2023). In this work, we analyze a broader class of neural ODEs where the dynamics function is a general nonlinear function, either time dependent or time independent, and is Lipschitz continuous with respect to the state variables. We showed that under this Lipschitz condition, the solutions to neural ODEs have solutions with bounded variations. Based on this observation, we establish generalization bounds for both time-dependent and time-independent cases and investigate how overparameterization and domain constraints influence these bounds. To our knowledge, this is the first derivation of generalization bounds for neural ODEs with general nonlinear dynamics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。