提出新型神经SDE积分方法,兼顾内存效率与梯度精度。
Explicit and Effectively Symmetric Schemes for Neural SDEs on Lie Groups
- 基于显式对称框架,设计可逆性近似但稳定的神经SDE求解器。
- 在欧氏空间上比现有可逆方法更稳定,大步长下表现优异。
- 适用于流形值问题,内存降低一个数量级,适合几何建模任务。
通过反向传播求解神经SDE的传统方法分为两类:离散化后优化(准确但内存高)和优化后离散化(内存恒定但速度慢、梯度有偏差)。代数可逆求解器可兼顾二者,但现有方法如可逆Heun在复杂模型和大步长下常不稳定,且非标准辅助状态结构难以推广至流形值SDE。本文基于最近提出的显式有效对称(EES)格式——一类稳定且近似可逆的显式龙格-库塔方法——将该框架扩展至随机微分方程,并证明其具有高效的威廉森2N存储实现。结合Bazavov无换位子构造,进一步将其提升至任意李群与齐性空间。据我们所知,这是首个在此类设置下的显式(近似)可逆积分器,首次为流形值问题启用可逆伴随法。在欧氏神经SDE基准测试中,新方法在刚性漂移和大步长下均优于其他可逆求解器;在流形值问题上,无换位子提升使内存减少达一个数量级。结果确立了有效对称积分作为统一、几何感知的神经SDE高效稳定训练基础。
原文摘要 · Abstract (English)
Backpropagation through (neural) SDE solvers is traditionally approached in two ways: discretise-then-optimise, which offers accurate gradients but incurs prohibitive memory costs; and optimise-then-discretise, which achieves constant memory cost by solving an auxiliary backward SDE, but suffers from slower evaluation and gradient approximation errors. Algebraically reversible solvers promise both memory efficiency and gradient accuracy, yet existing methods such as Reversible Heun are often unstable under complex models and large step sizes, and their non-standard auxiliary-state structure obstructs extension to manifold-valued SDEs. Building on the recently introduced Explicit and Effectively Symmetric (EES) schemes - a class of stable, near-reversible explicit Runge--Kutta methods - we address both limitations of existing schemes. We extend EES schemes from ODEs to SDEs and show that they admit an efficient Williamson 2N-storage realisation. Bazavov's commutator-free construction then lifts these schemes to arbitrary Lie groups and homogeneous spaces. To our knowledge, this is the first explicit (near-)reversible integrator in this setting, unlocking the reversible adjoint approach for manifold-valued problems. On Euclidean neural SDE benchmarks, our schemes improve stability under stiff drift and large steps compared with other reversible solvers, while the commutator-free lift reduces memory by up to an order of magnitude on manifold-valued problems versus other baselines. These results establish effectively symmetric integration as a unified, geometry-aware foundation for memory-efficient and stable training of neural SDEs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。