提出新型自然梯度方法，加速PINNs训练并提升精度
ANaGRAM: A Natural Gradient Relative to Adapted Model for efficient PINNs learning
- 基于微分几何构造新自然梯度算法，复杂度为min(P²S, S²P)
- 训练速度显著提升，且理论证明与格林函数存在关联
- 适合研究物理信息神经网络的高效优化者
近年来，物理信息神经网络（PINNs）因其在求解偏微分方程驱动系统中的潜力而受到广泛关注，尤其适用于数据同化任务。然而该方法仍处于发展初期，存在诸多未被充分理解的缺陷与失败案例。本文提出一种针对PINNs的自然梯度方法，有效加速训练过程并提高精度。基于对问题微分几何结构的深入分析，本文提出两项贡献：(i) 一种新的自然梯度算法，其计算复杂度为min(P²S, S²P)，其中P为参数数量，S为批大小；(ii) 一种数学上严谨的PINNs问题重表述，使自然梯度可应用于该框架，并证明了其与格林函数理论的理论联系。
原文摘要 · Abstract (English)
In the recent years, Physics Informed Neural Networks (PINNs) have received strong interest as a method to solve PDE driven systems, in particular for data assimilation purpose. This method is still in its infancy, with many shortcomings and failures that remain not properly understood. In this paper we propose a natural gradient approach to PINNs which contributes to speed-up and improve the accuracy of the training. Based on an in depth analysis of the differential geometric structures of the problem, we come up with two distinct contributions: (i) a new natural gradient algorithm that scales as $\min(P^2S, S^2P)$, where $P$ is the number of parameters, and $S$ the batch size; (ii) a mathematically principled reformulation of the PINNs problem that allows the extension of natural gradient to it, with proved connections to Green's function theory.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。