用同步优化方法加速神经微分方程训练，提升收敛速度。
Training Neural ODEs Using Fully Discretized Simultaneous Optimization
- 将微分方程离散化为协同优化问题，同时求解网络参数与节点值。
- 在Van der Pol振子上实现比传统方法更快的收敛，计算效率显著提升。
- 适用于大规模数据训练，适合需要高效建模连续动态的场景。
神经微分方程（Neural ODEs）通过神经网络表示连续时间动态，在建模与控制任务中表现优异。但传统训练需每轮求解微分方程，计算开销大。本文提出一种基于配点法的完全离散化协同优化方法，使用IPOPT求解器同时优化配点系数与神经网络参数。以Van der Pol振子为例，实验表明其收敛速度优于传统方法。此外，引入基于交替方向乘子法（ADMM）的分解框架，有效协调批数据间的子模型。结果表明，该方法具备构建高效（基于配点）神经ODE训练流程的巨大潜力。
原文摘要 · Abstract (English)
Neural Ordinary Differential Equations (Neural ODEs) represent continuous-time dynamics with neural networks, offering advancements for modeling and control tasks. However, training Neural ODEs requires solving differential equations at each epoch, leading to high computational costs. This work investigates simultaneous optimization methods as a faster training alternative. In particular, we employ a collocation-based, fully discretized formulation and use IPOPT--a solver for large-scale nonlinear optimization--to simultaneously optimize collocation coefficients and neural network parameters. Using the Van der Pol Oscillator as a case study, we demonstrate faster convergence compared to traditional training methods. Furthermore, we introduce a decomposition framework utilizing Alternating Direction Method of Multipliers (ADMM) to effectively coordinate sub-models among data batches. Our results show significant potential for (collocation-based) simultaneous Neural ODE training pipelines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。