通过分析核矩阵特征值，量化神经网络求解PDE的训练难度并加速收敛。
Quantifying Training Difficulty and Accelerating Convergence in Neural Network-Based PDE Solvers
- 用有效秩衡量初始化对训练难易的影响。
- 发现PoU和方差缩放可提升有效秩，加快误差下降速度。
- 在PINN、Deep Ritz等框架中均验证了加速效果，适合求解复杂PDE的研究者。
基于神经网络的偏微分方程（PDE）求解方法在科学与工程领域日益重要，尤其适用于复杂域或融合实验数据的情形。此类方法利用神经网络作为基函数逼近PDE解，但训练过程常面临收敛困难、精度受限的问题。本文聚焦于神经网络型PDE求解器的训练动态，重点研究初始化策略的影响。通过分析核矩阵的特征值分布，并引入有效秩来量化训练难度——有效秩越大，训练误差收敛越快。理论分析与数值实验表明，两种初始化方法：分片单位分解（Partition of Unity, PoU）和方差缩放（Variance Scaling, VS），能显著提升有效秩，从而加速训练误差收敛。进一步在典型PDE求解框架（如PINN、Deep Ritz、DeepOnet）上的全面实验验证了该结论的一致性，结果与理论预测高度吻合。
原文摘要 · Abstract (English)
Neural network-based methods have emerged as powerful tools for solving partial differential equations (PDEs) in scientific and engineering applications, particularly when handling complex domains or incorporating empirical data. These methods leverage neural networks as basis functions to approximate PDE solutions. However, training such networks can be challenging, often resulting in limited accuracy. In this paper, we investigate the training dynamics of neural network-based PDE solvers with a focus on the impact of initialization techniques. We assess training difficulty by analyzing the eigenvalue distribution of the kernel and apply the concept of effective rank to quantify this difficulty, where a larger effective rank correlates with faster convergence of the training error. Building upon this, we discover through theoretical analysis and numerical experiments that two initialization techniques, partition of unity (PoU) and variance scaling (VS), enhance the effective rank, thereby accelerating the convergence of training error. Furthermore, comprehensive experiments using popular PDE-solving frameworks, such as PINN, Deep Ritz, and the operator learning framework DeepOnet, confirm that these initialization techniques consistently speed up convergence, in line with our theoretical findings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。