arXiv:2508.02692cs.CEcs.LG2025-08被引 1

提出新型损失函数,让基于优化的偏微分方程求解更快更稳。

Overcoming the Loss Conditioning Bottleneck in Optimization-Based PDE Solvers: A Novel Well-Conditioned Loss Function

  • 设计稳定梯度残差损失,动态调节问题条件数。
  • 在ODIL框架下收敛速度比MSE快数个数量级。
  • 适用于神经网络和传统离散方法,提升求解效率。

基于优化的偏微分方程求解器近年来受到关注,包括直接在离散变量上定义损失的ODIL和通过神经网络代理间接定义损失的PINNs。然而,这类方法常因收敛慢而被视为低效,本文从理论上揭示其根源在于使用均方误差(MSE)损失会隐式形成正规方程,平方条件数,严重恶化优化性能。为此,提出新型稳定梯度残差(SGR)损失,通过调节权重参数,灵活控制原系统与正规方程间的条件数,且在极限情况下退化为MSE。系统性地在ODIL和PINNs框架中(使用数值或自动微分)测试其收敛性和稳定性,并对比经典迭代求解器。数值实验表明,在ODIL框架中,SGR损失相比MSE实现数个数量级的加速;在PINNs框架中,即使面对高非线性神经网络,SGR仍持续优于MSE。理论与实证结果弥合了经典求解器与优化型求解器间的性能差距,凸显损失条件数的关键作用,为高效求解器设计提供核心洞见。

原文摘要 · Abstract (English)

Optimization-based PDE solvers that minimize scalar loss functions have gained increasing attention in recent years. These methods either define the loss directly over discrete variables, as in Optimizing a Discrete Loss (ODIL), or indirectly through a neural network surrogate, as in Physics-Informed Neural Networks (PINNs). However, despite their promise, such methods often converge much more slowly than classical iterative solvers and are commonly regarded as inefficient. This work provides a theoretical insight, attributing the inefficiency to the use of the mean squared error (MSE) loss, which implicitly forms the normal equations, squares the condition number, and severely impairs optimization. To address this, we propose a novel Stabilized Gradient Residual (SGR) loss. By tuning a weight parameter, it flexibly modulates the condition number between the original system and its normal equations, while reducing to the MSE loss in the limiting case. We systematically benchmark the convergence behavior and optimization stability of the SGR loss within both the ODIL framework and PINNs-employing either numerical or automatic differentiation-and compare its performance against classical iterative solvers. Numerical experiments on a range of benchmark problems demonstrate that, within the ODIL framework, the proposed SGR loss achieves orders-of-magnitude faster convergence than the MSE loss. Further validation within the PINNs framework shows that, despite the high nonlinearity of neural networks, SGR consistently outperforms the MSE loss. These theoretical and empirical findings help bridge the performance gap between classical iterative solvers and optimization-based solvers, highlighting the central role of loss conditioning, and provide key insights for the design of more efficient PDE solvers.

偏微分方程优化求解损失函数数值方法

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。