arXiv:2607.05280cs.LG2026-07被引 1

提升神经控制微分方程训练效率,实现快速高精度时序建模。

Advances in Neural Controlled Differential Equations

论文配图:Advances in Neural Controlled Differential Equations
图 1 · 摘自论文原文
  • 用对数龙格-库塔法加速训练,降低计算开销。
  • 线性向量场设计支持并行计算,速度提升百倍以上。
  • 适合处理不规则采样时序数据,适用于医疗、金融等领域。

许多现实世界系统连续演化,但多数机器学习模型将时间序列视为离散序列。连续时间方法则将时间序列视为潜在输入路径的采样,天然适用于不规则或过采样的数据。其中,神经控制微分方程(NCDEs)是一类表达能力最强的模型,通过神经网络参数化向量场,并通过求解受输入路径驱动的动力系统来演化隐藏状态。然而,传统NCDEs使用非线性向量场,导致前向传播计算昂贵且本质串行,限制了其可扩展性和实用性。本文通过三项互补贡献推进NCDEs的训练与可扩展性:第一,基于神经粗糙微分方程,提出Log-NCDEs,利用对数-ODE方法高效近似训练中的解,提升计算速度与实证性能;第二,引入线性NCDEs,以线性向量场替代非线性,实现闭式解与并行时间计算,同时保持理论表达能力;第三,结构化线性NCDEs采用结构化线性向量场,进一步提升效率,兼顾理论表达力与实证表现。三者结合使单步训练时间减少高达三个数量级,在多个时间序列基准上达到当前最优性能。

原文摘要 · Abstract (English)

Many real-world systems evolve continuously, yet most machine learning models interpret time series as discrete sequences. Continuous-time approaches instead treat time series as samples from an underlying input path, a formulation that naturally accommodates irregularly sampled or oversampled data. Among these, Neural Controlled Differential Equations (NCDEs) are a maximally expressive class of models that parametrise a vector field using a neural network and evolve their hidden state by solving a dynamical system driven by the input path. NCDEs typically use a non-linear vector field, so their expressive power and continuous-time flexibility come at the cost of a forward pass that is both computationally expensive and inherently sequential, limiting their scalability and practical applicability. This thesis advances the training and scalability of NCDEs through three complementary contributions. First, building on neural rough differential equations, Log-NCDEs apply the Log-ODE method to efficiently approximate an NCDE's solution during training, improving both computational speed and empirical performance. Second, Linear NCDEs replace the non-linear vector field with a linear one, enabling closed-form solutions and parallel-in-time computation without sacrificing theoretical expressivity. Third, Structured Linear NCDEs use structured linear vector fields to further enhance efficiency while maintaining theoretical expressiveness and empirical performance. Collectively, these methods reduce the time per training step for an NCDE by up to three orders of magnitude while achieving state-of-the-art performance across diverse time series benchmarks.

时序建模微分方程高效训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。