提出可逆神经ODE求解器，实现高效精确的梯度计算。
Efficient, Accurate and Stable Gradients for Neural ODEs
- 采用代数可逆的微分方程求解方法，支持精确梯度传播。
- 相比递归检查点法，时间与内存开销显著降低。
- 高阶精度且数值稳定，适合对稳定性要求高的模型训练。
训练神经微分方程需要通过求解微分方程进行反向传播。当前最先进的方法是递归检查点法，可在重计算与内存消耗之间取得平衡。本文提出一类代数可逆的神经微分方程求解器，显著优于递归检查点法在时间和内存上的表现。所提出的可逆求解器能计算精确梯度，具备高阶精度和数值稳定性，严格改进了以往可逆架构的性能。
原文摘要 · Abstract (English)
Training Neural ODEs requires backpropagating through an ODE solve. The state-of-the-art backpropagation method is recursive checkpointing that balances recomputation with memory cost. Here, we introduce a class of algebraically reversible ODE solvers that significantly improve upon both the time and memory cost of recursive checkpointing. The reversible solvers presented calculate exact gradients, are high-order and numerically stable -- strictly improving on previous reversible architectures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。