用硬约束优化提升复杂PDE求解精度,误差降低1-3个数量级。
Physics-Informed Neural Networks with Trust-Region Sequential Quadratic Programming
- 基于信赖域的二次规划,将物理约束转为硬约束,避免软约束失效
- 相比传统方法误差降低1至3个数量级,显著提升复杂PDE求解精度
- 引入预训练和拟牛顿更新,兼顾稳定性与计算效率,适合科研与工程应用
物理信息神经网络(PINNs)是科学机器学习的重要进展,通过在经验损失函数中嵌入物理知识作为软约束,并利用现有机器学习方法进行训练。然而,近期研究发现PINNs在求解较复杂偏微分方程(PDEs)时可能失败。本文提出一种新型硬约束深度学习方法——信赖域序列二次规划(trSQP-PINN),以应对该问题。与直接优化惩罚软约束损失的PINNs不同,本方法对硬约束损失进行线性-二次逼近,并利用软约束损失自适应调整信赖域半径。仅在信赖域内信任模型近似并执行更新,有效缓解了PINNs的病态性问题。同时,通过拟牛顿更新克服二阶SQP方法的计算瓶颈,并引入简单预训练步骤进一步提升训练效率。大量实验表明,trSQP-PINN相比现有硬约束方法(如罚方法、增广拉格朗日法)显著提升解的精度,误差降低达1-3个数量级。此外,预训练策略对其他硬约束方法亦具普适有效性,且方法对问题参数与算法调参均表现出强鲁棒性。
原文摘要 · Abstract (English)
Physics-Informed Neural Networks (PINNs) represent a significant advancement in Scientific Machine Learning (SciML), which integrate physical domain knowledge into an empirical loss function as soft constraints and apply existing machine learning methods to train the model. However, recent research has noted that PINNs may fail to learn relatively complex Partial Differential Equations (PDEs). This paper addresses the failure modes of PINNs by introducing a novel, hard-constrained deep learning method -- trust-region Sequential Quadratic Programming (trSQP-PINN). In contrast to directly training the penalized soft-constrained loss as in PINNs, our method performs a linear-quadratic approximation of the hard-constrained loss, while leveraging the soft-constrained loss to adaptively adjust the trust-region radius. We only trust our model approximations and make updates within the trust region, and such an updating manner can overcome the ill-conditioning issue of PINNs. We also address the computational bottleneck of second-order SQP methods by employing quasi-Newton updates for second-order information, and importantly, we introduce a simple pretraining step to further enhance training efficiency of our method. We demonstrate the effectiveness of trSQP-PINN through extensive experiments. Compared to existing hard-constrained methods for PINNs, such as penalty methods and augmented Lagrangian methods, trSQP-PINN significantly improves the accuracy of the learned PDE solutions, achieving up to 1-3 orders of magnitude lower errors. Additionally, our pretraining step is generally effective for other hard-constrained methods, and experiments have shown the robustness of our method against both problem-specific parameters and algorithm tuning parameters.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。