无需模拟即可训练神经微分方程,提升效率与稳定性。
Simulation-Free Training of Neural ODEs on Paired Data
- 在数据嵌入空间中直接匹配流,避免轨迹交叉问题。
- 相比传统方法,函数求值次数显著减少,性能更优。
- 适合需要高效推理的回归与分类任务,尤其关注速度。
本文研究了无需模拟的神经常微分方程(NODEs)在成对数据间确定性映射学习中的训练方法。尽管NODEs可类比为连续深度残差网络,但其在典型监督学习任务中应用较少,主要受限于求解器所需大量函数评估及梯度估计的数值不稳定性。为此,我们采用流匹配框架实现无模拟训练,直接将参数化动力学函数拟合到预定义的目标速度场。然而,直接在原始数据间应用流匹配常导致流定义不良,破坏数据对耦合关系(如轨迹交叉)。为此,我们提出在联合学习的数据嵌入空间中进行流匹配,确保流的有效性且更易学习。实验表明,该方法在回归与分类任务上均优于现有NODEs,函数评估次数大幅降低。代码已开源:https://github.com/seminkim/simulation-free-node。
原文摘要 · Abstract (English)
In this work, we investigate a method for simulation-free training of Neural Ordinary Differential Equations (NODEs) for learning deterministic mappings between paired data. Despite the analogy of NODEs as continuous-depth residual networks, their application in typical supervised learning tasks has not been popular, mainly due to the large number of function evaluations required by ODE solvers and numerical instability in gradient estimation. To alleviate this problem, we employ the flow matching framework for simulation-free training of NODEs, which directly regresses the parameterized dynamics function to a predefined target velocity field. Contrary to generative tasks, however, we show that applying flow matching directly between paired data can often lead to an ill-defined flow that breaks the coupling of the data pairs (e.g., due to crossing trajectories). We propose a simple extension that applies flow matching in the embedding space of data pairs, where the embeddings are learned jointly with the dynamic function to ensure the validity of the flow which is also easier to learn. We demonstrate the effectiveness of our method on both regression and classification tasks, where our method outperforms existing NODEs with a significantly lower number of function evaluations. The code is available at https://github.com/seminkim/simulation-free-node.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。