证明了基于龙格-库塔法的NeuralODE在最大似然训练下是可靠的机器学习算法。
Numerical and statistical analysis of NeuralODE with Runge-Kutta time integration
- 采用二阶龙格-库塔法数值求解NeuralODE微分方程流
- 理论证明其在统计与数值层面均具收敛性，属于近似正确学习算法
- 适用于对生成模型的稳定性与可学习性有严格要求的研究者
NeuralODE是一种基于双射映射推动简单源分布的生成机器学习方法，其映射由常微分方程的流给出。利用Liouville公式，可高效计算推动测度的对数密度，从而通过最大似然法训练，使推动测度与数据目标测度之间的Kullback-Leibler散度最小化。本文针对一类通用目标测度，详细分析了基于最大似然的经验风险最小化的一致性。不同于以往工作，不仅引入统计学习理论，还对基于二阶龙格-库塔（RK）时间积分的NeuralODE算法进行了深入数值分析。结合深度ReQU网络的通用逼近性、RK格式的稳定性和收敛速率、度量熵及浓度不等式，我们证明了NeuralODE是一个可能近似正确（PAC）的学习算法。
原文摘要 · Abstract (English)
NeuralODE is one example for generative machine learning based on the push forward of a simple source measure with a bijective mapping, which in the case of NeuralODE is given by the flow of a ordinary differential equation. Using Liouville's formula, the log-density of the push forward measure is easy to compute and thus NeuralODE can be trained based on the maximum Likelihood method such that the Kulback-Leibler divergence between the push forward through the flow map and the target measure generating the data becomes small. In this work, we give a detailed account on the consistency of Maximum Likelihood based empirical risk minimization for a generic class of target measures. In contrast to prior work, we do not only consider the statistical learning theory, but also give a detailed numerical analysis of the NeuralODE algorithm based on the 2nd order Runge-Kutta (RK) time integration. Using the universal approximation theory for deep ReQU networks, the stability and convergence rated for the RK scheme as well as metric entropy and concentration inequalities, we are able to prove that NeuralODE is a probably approximately correct (PAC) learning algorithm.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。