提出新方法训练神经分数阶微分方程，大幅降低内存占用。
Efficient Training of Neural Fractional-Order Differential Equation via Adjoint Backpropagation
- 通过反向求解扩展的分数阶方程实现高效反向传播
- 在多个任务中性能接近基线模型，计算开销显著降低
- 适合需要大规模训练的复杂系统建模场景
分数阶微分方程（FDEs）将微分算子阶数从整数扩展至实数，增强了对具有非局部特性的复杂动态系统的建模灵活性。近年来，FDE与深度学习的结合催生了新型模型，在图表示学习等任务中展现出潜力。然而，现有神经FDE训练主要依赖数值求解器的前向过程直接求导，导致内存消耗和计算复杂度高，尤其在大规模应用中问题突出。为此，我们提出一种可扩展的伴随反向传播方法，通过反向时间求解扩展的FDE，显著降低内存需求。该方法构建了一个实用的神经FDE工具箱，具备广泛的应用前景。我们在多个任务中验证了其有效性，性能接近基线模型，同时大幅减少计算开销。
原文摘要 · Abstract (English)
Fractional-order differential equations (FDEs) enhance traditional differential equations by extending the order of differential operators from integers to real numbers, offering greater flexibility in modeling complex dynamical systems with nonlocal characteristics. Recent progress at the intersection of FDEs and deep learning has catalyzed a new wave of innovative models, demonstrating the potential to address challenges such as graph representation learning. However, training neural FDEs has primarily relied on direct differentiation through forward-pass operations in FDE numerical solvers, leading to increased memory usage and computational complexity, particularly in large-scale applications. To address these challenges, we propose a scalable adjoint backpropagation method for training neural FDEs by solving an augmented FDE backward in time, which substantially reduces memory requirements. This approach provides a practical neural FDE toolbox and holds considerable promise for diverse applications. We demonstrate the effectiveness of our method in several tasks, achieving performance comparable to baseline models while significantly reducing computational overhead.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。