用64位精度可彻底解决物理信息神经网络的训练失败问题
FP64 is All You Need: Rethinking Failure Modes in Physics-Informed Neural Networks
- 将数值精度从FP32升级至FP64，避免优化器过早收敛
- 在无任何改进的情况下，原生PINN在FP64下可完全避免失败模式
- 适合关注神经网络求解微分方程稳定性的研究者
物理信息神经网络（PINNs）常出现残差损失收敛但解误差仍大的现象，传统认为是局部极小值导致。本文指出真正原因是标准FP32精度下LBFGS优化器过早满足收敛条件，使网络陷入虚假失败阶段。仅升级至FP64即可恢复优化过程，使原始PINN无需任何修改即可成功求解偏微分方程。该发现将失败模式重新定义为精度引发的停滞，揭示了未收敛、失败、成功三阶段训练动态，且其边界随数值精度变化。研究强调高精度算术是可靠神经网络求解微分方程的关键。
原文摘要 · Abstract (English)
Physics Informed Neural Networks (PINNs) often exhibit failure modes in which the PDE residual loss converges while the solution error stays large, a phenomenon traditionally blamed on local optima separated from the true solution by steep loss barriers. We challenge this understanding by demonstrate that the real culprit is insufficient arithmetic precision: with standard FP32, the LBFGS optimizer prematurely satisfies its convergence test, freezing the network in a spurious failure phase. Simply upgrading to FP64 rescues optimization, enabling vanilla PINNs to solve PDEs without any failure modes. These results reframe PINN failure modes as precision induced stalls rather than inescapable local minima and expose a three stage training dynamic unconverged, failure, success whose boundaries shift with numerical precision. Our findings emphasize that rigorous arithmetic precision is the key to dependable PDE solving with neural networks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。