用控制理论方法高效训练神经随机微分方程，提升时序不确定性建模效率。
Efficient Training of Neural SDEs Using Stochastic Optimal Control
- 将控制项分解为线性与非线性部分，线性部分由最优控制直接求解。
- 训练成本降低，收敛速度加快，且保持模型表达能力。
- 适合需要高效时序建模与不确定性估计的研究者。
我们提出一种分层、受控制理论启发的变分推断（VI）方法，用于神经随机微分方程（SDEs）。尽管神经SDE在时间序列不确定性推理中前景广阔，但其变分推断因最大化证据下界（ELBO）的迭代特性而计算成本高。本文将控制项分解为线性与残差非线性部分，利用随机最优控制推导出线性SDE的最优控制项。通过神经网络建模非线性部分，实现了无需牺牲表达力的高效训练。由于线性控制项为最优且无需学习，训练初始成本更低，我们观察到收敛速度显著提升。
原文摘要 · Abstract (English)
We present a hierarchical, control theory inspired method for variational inference (VI) for neural stochastic differential equations (SDEs). While VI for neural SDEs is a promising avenue for uncertainty-aware reasoning in time-series, it is computationally challenging due to the iterative nature of maximizing the ELBO. In this work, we propose to decompose the control term into linear and residual non-linear components and derive an optimal control term for linear SDEs, using stochastic optimal control. Modeling the non-linear component by a neural network, we show how to efficiently train neural SDEs without sacrificing their expressive power. Since the linear part of the control term is optimal and does not need to be learned, the training is initialized at a lower cost and we observe faster convergence.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。