将深度神经网络视为离散动力系统,揭示其与物理学习的深层联系。
Deep Neural Networks as Discrete Dynamical Systems: Implications for Physics-Informed Learning
- 用神经积分方程类比DNN层间演进,视作离散动力系统
- PINN逼近结果与经典数值解相似,但路径不同且参数更密集
- 适合高维问题,但解释性差、计算开销大
我们重新审视前馈深度神经网络(DNN)与由神经积分方程及其对应偏微分方程(PDE)形式导出的离散动力系统的类比关系。通过对比伯格斯方程和埃克尔方程的数值解/精确解与基于物理信息神经网络(PINN)获得的结果,表明PINN学习提供了一种不同于标准数值离散化的计算路径,却逼近了相同的系统动态。在此框架下,DNN可被理解为逐层演进的离散动力系统,其演化趋近于吸引子,且多种参数配置可能产生相近解,反映出逆映射的退化性。与有限差分(FD)方法所关联的结构化算子不同,PINN学习的是稠密参数表示,不直接对应传统离散化模板。这种分布式表示通常涉及更多参数,导致解释性降低和计算成本增加,但在经典网格方法变得不切实际的高维场景中,可能具备优势。
原文摘要 · Abstract (English)
We revisit the analogy between feed-forward deep neural networks (DNNs) and discrete dynamical systems derived from neural integral equations and their corresponding partial differential equation (PDE) forms. A comparative analysis between the numerical/exact solutions of the Burgers' and Eikonal equations, and the same obtained via PINNs is presented. We show that PINN learning provides a different computational pathway compared to standard numerical discretization in approximating essentially the same underlying dynamics of the system. Within this framework, DNNs can be interpreted as discrete dynamical systems whose layer-wise evolution approaches attractors, and multiple parameter configurations may yield comparable solutions, reflecting the degeneracy of the inverse mapping. In contrast to the structured operators associated with finite-difference (FD) procedures, PINNs learn dense parameter representations that are not directly associated with classical discretization stencils. This distributed representation generally involves a larger number of parameters, leading to reduced interpretability and increased computational cost. However, the additional flexibility of such representations may offer advantages in high-dimensional settings where classical grid-based methods become impractical.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。