针对物理信息神经网络的梯度冲突问题,提出按冲突类型选择不同干预策略。
Per-Loss Adapters for Gradient Conflict in Physics-Informed Neural Networks

- 按梯度冲突类型划分干预方案:方向冲突用损失专属子空间,幅度失衡用加权,轻微冲突无需处理。
- 在60多个偏微分方程任务中,组合使用适配器与重加权显著提升性能,尤其在高维和多物理场问题中。
- 提出诊断先行框架,通过1000步无修改训练判断是否需要干预,适合研究者快速评估模型瓶颈。
物理信息神经网络(PINNs)通过最小化多个物理和数据导出的损失来训练单一神经网络,但这些损失的梯度常发生冲突,导致优化停滞。现有方法通常采用标量损失平衡或全参数空间梯度手术,但未明确哪种干预更合适。本文表明,PINN的梯度冲突并非统一故障模式,而是存在不同冲突类型,每类对应不同干预方式。持续方向冲突需损失索引的参数子空间,幅度不平衡则更适合标量重加权,而低频或瞬时冲突可能无需额外处理。为此,我们提出诊断优先框架:先运行1000步无修改的PINN,若需干预,则为每个损失添加一个低秩适配器,构建与共享主干相连的损失专属参数子空间,提供直接梯度通路。在超过60种偏微分方程配置中,包括前向、反向、多物理场、参数变化及高达50维的问题,标准前向K=3基准和自然K=4热弹性系统中,适配器结合重加权表现显著提升。而K=3反向问题及自然K=5、K=6多物理场系统主要受幅度主导,通常仅重加权即可,全参数空间梯度手术在异质参数空间中可能失效。
原文摘要 · Abstract (English)
Physics-informed neural networks (PINNs) train a single neural approximation by minimizing multiple physics- and data-derived losses, but the gradients of these losses often interfere and can stall optimization. Existing remedies typically treat this pathology either through scalar loss balancing or full-parameter-space gradient surgery, leaving it unclear which intervention is most appropriate. We show that PINN gradient conflict is not a uniform failure mode with one universal remedy. Instead, we identify distinct PINN gradient-conflict regimes, each associated with a different intervention class. Persistent directional conflict may require separate loss-indexed parameter subspaces, magnitude imbalance often favors scalar reweighting, and low or transient conflict may require no extra mitigation. To select between scalar reweighting and a lightweight architectural intervention, we propose a diagnostic-first framework. It profiles a 1000-step unmodified PINN run and, when intervention is warranted, uses one low-rank adapter per loss to create explicit loss-indexed parameter subspaces attached to a shared PINN trunk, providing each loss with a direct gradient pathway. Across more than 60 PDE configurations, including forward, inverse, multi-physics, parameter-varying, and high-dimensional problems up to 50D, persistent directional conflict dominates standard forward $K=3$ benchmarks and a natural $K=4$ thermoelastic system, where adapters combined with reweighting yield significant improvements. In contrast, $K=3$ inverse problems and natural $K=5$ and $K=6$ multi-physics systems are largely magnitude-dominated and often favor reweighting alone, while full-parameter-space gradient surgery can fail on heterogeneous parameter spaces.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。