用符号神经网络学习复杂系统的动态演化规律。
Symbolic Neural Ordinary Differential Equations
- 分三阶段训练:预训练+微调+残差建模,融合符号与连接主义优势。
- 在偏微分方程系统中实现分辨率不变建模,提升泛化能力。
- 适合需要可解释性与外推能力的科学建模任务,如方程发现、控制等。
微分方程广泛用于描述自然界和工程中参数随时间演化的复杂动力系统。有效学习从参数函数到系统动态的映射关系具有重要意义。本文提出一种新型符号连续深度神经网络学习框架——符号神经常微分方程(SNODEs),以高效且准确地学习复杂系统的内在动态。该框架包含三个阶段:首先通过梯度流匹配策略预训练一个预定义的符号神经网络;随后使用神经常微分方程(Neural ODEs)进行微调;最后构建通用神经网络捕捉残差。在此过程中,我们通过傅里叶分析将SNODEs框架应用于偏微分方程系统,实现了分辨率不变建模。该框架融合符号与连接主义优势,具备通用逼近定理,相比现有最先进方法显著提升了可解释性与外推能力。我们在多个代表性复杂系统上进行了实验验证。因此,该框架可进一步应用于系统分岔、控制、重构与预测,以及新方程的发现等广泛科学问题。
原文摘要 · Abstract (English)
Differential equations are widely used to describe complex dynamical systems with evolving parameters in nature and engineering. Effectively learning a family of maps from the parameter function to the system dynamics is of great significance. In this study, we propose a novel learning framework of symbolic continuous-depth neural networks, termed Symbolic Neural Ordinary Differential Equations (SNODEs), to effectively and accurately learn the underlying dynamics of complex systems. Specifically, our learning framework comprises three stages: initially, pre-training a predefined symbolic neural network via a gradient flow matching strategy; subsequently, fine-tuning this network using Neural ODEs; and finally, constructing a general neural network to capture residuals. In this process, we apply the SNODEs framework to partial differential equation systems through Fourier analysis, achieving resolution-invariant modeling. Moreover, this framework integrates the strengths of symbolism and connectionism, boasting a universal approximation theorem while significantly enhancing interpretability and extrapolation capabilities relative to state-of-the-art baseline methods. We demonstrate this through experiments on several representative complex systems. Therefore, our framework can be further applied to a wide range of scientific problems, such as system bifurcation and control, reconstruction and forecasting, as well as the discovery of new equations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。