用自然梯度加速随机微分方程的隐变量推断,提升稳定性与速度。
SING: SDE Inference via Natural Gradients
- 基于自然梯度变分推断,利用模型几何结构优化推断过程
- 在多个数据集上实现更快收敛和更准确的状态估计
- 适合缺乏先验知识、非共轭结构的复杂动态系统研究
潜在随机微分方程(SDE)模型是从未标记数据中无监督发现动态系统的有力工具,应用涵盖工程与神经科学。在这些复杂领域中,隐变量路径的精确后验推断通常不可行,促使采用近似方法如变分推断(VI)。然而现有用于潜在SDE的VI方法常存在收敛慢与数值不稳定的缺陷。本文提出SING(SDE Inference via Natural Gradients),通过自然梯度变分推断高效利用模型与变分后验的内在几何结构。SING通过近似不可积分项并并行化时间维度计算,实现快速可靠的隐变量推断。我们提供了理论保证,证明SING可近似优化目标的连续时间不可行目标。此外,更优的状态推断显著提升了非线性漂移函数的估计精度,例如在高斯过程SDE模型中。SING在多种数据集上优于先前方法,在自由活动动物神经动力学建模等挑战性任务中表现突出。结果表明SING是复杂动态系统中精确推断的有力工具,尤其适用于先验知识有限且结构非共轭的情形。
原文摘要 · Abstract (English)
Latent stochastic differential equation (SDE) models are important tools for the unsupervised discovery of dynamical systems from data, with applications ranging from engineering to neuroscience. In these complex domains, exact posterior inference of the latent state path is typically intractable, motivating the use of approximate methods such as variational inference (VI). However, existing VI methods for inference in latent SDEs often suffer from slow convergence and numerical instability. We propose SDE Inference via Natural Gradients (SING), a method that leverages natural gradient VI to efficiently exploit the underlying geometry of the model and variational posterior. SING enables fast and reliable inference in latent SDE models by approximating intractable integrals and parallelizing computations in time. We provide theoretical guarantees that SING approximately optimizes the intractable, continuous-time objective of interest. Moreover, we demonstrate that better state inference enables more accurate estimation of nonlinear drift functions using, for example, Gaussian process SDE models. SING outperforms prior methods in state inference and drift estimation on a variety of datasets, including a challenging application to modeling neural dynamics in freely behaving animals. Altogether, our results illustrate the potential of SING as a tool for accurate inference in complex dynamical systems, especially those characterized by limited prior knowledge and non-conjugate structure.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。