arXiv:2410.03973cs.LGstat.ML2024-10ICLR被引 5

提出新方法高效训练神经随机微分方程,速度提升显著。

Efficient Training of Neural Stochastic Differential Equations by Matching Finite Dimensional Distributions

  • 基于马尔可夫性质设计有限维分布匹配训练目标
  • 训练复杂度从O(D²)降至O(D),D为离散步数
  • 无需对抗训练或求解线性PDE,适合高维连续过程建模

神经随机微分方程(Neural SDEs)作为连续随机过程的无网格生成模型,在金融、物理和生物等领域具有重要应用。现有最优方法依赖对抗训练(如GAN)或签名核距离度量,但前者存在不稳定性、模式崩溃等问题,后者需求解线性偏微分方程并反向传播梯度,计算复杂度随离散步数呈平方增长。本文识别出一类严格合适的评分规则,用于比较连续马尔可夫过程,由此提出有限维分布匹配(FDM)训练方法。该方法利用SDE的马尔可夫特性,避免了签名核的计算开销,将每轮训练复杂度从O(D²)降低至O(D),其中D为过程的离散步数。实验表明,FDM在生成质量与计算效率上均优于现有方法。

原文摘要 · Abstract (English)

Neural Stochastic Differential Equations (Neural SDEs) have emerged as powerful mesh-free generative models for continuous stochastic processes, with critical applications in fields such as finance, physics, and biology. Previous state-of-the-art methods have relied on adversarial training, such as GANs, or on minimizing distance measures between processes using signature kernels. However, GANs suffer from issues like instability, mode collapse, and the need for specialized training techniques, while signature kernel-based methods require solving linear PDEs and backpropagating gradients through the solver, whose computational complexity scales quadratically with the discretization steps. In this paper, we identify a novel class of strictly proper scoring rules for comparing continuous Markov processes. This theoretical finding naturally leads to a novel approach called Finite Dimensional Matching (FDM) for training Neural SDEs. Our method leverages the Markov property of SDEs to provide a computationally efficient training objective. This scoring rule allows us to bypass the computational overhead associated with signature kernels and reduces the training complexity from $O(D^2)$ to $O(D)$ per epoch, where $D$ represents the number of discretization steps of the process. We demonstrate that FDM achieves superior performance, consistently outperforming existing methods in terms of both computational efficiency and generative quality.

神经SDE生成模型随机微分方程高效训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。