解决疫情模型训练中的梯度冲突问题,提升稳定性和精度。
Resolving gradient pathology in physics-informed epidemiological models
- 用梯度方向相似性动态调节物理约束权重,避免优化冲突。
- 在复杂疫情系统中实现更准确的参数估计和更快收敛。
- 适合需要高精度建模的流行病学研究者使用。
物理信息神经网络(PINNs)被广泛用于数学流行病学,连接嘈杂的临床数据与如易感-暴露-感染-移除(SEIR)等常微分方程模型。然而,由于数据损失与物理残差的目标相互竞争,训练常不稳定。近期研究表明,数据梯度与物理梯度方向冲突会导致收敛缓慢或优化死锁。现有方法通过平衡梯度幅值或投影来缓解,但效果有限。本文提出冲突门控梯度缩放(CGGS),利用数据与物理梯度间的余弦相似度动态调节惩罚权重。当梯度方向冲突大时,抑制物理约束以优先拟合数据;方向一致时则恢复约束。理论证明该机制可保持平滑非凸目标下$O(1/T)$的收敛率,而固定权重或幅值平衡法在冲突时失效。实验证明,该方法能自动诱导课程学习效应,在刚性流行病系统中显著优于基于幅值的方法,提升峰值恢复能力与收敛速度。
原文摘要 · Abstract (English)
Physics-informed neural networks (PINNs) are increasingly used in mathematical epidemiology to bridge the gap between noisy clinical data and compartmental models, such as the susceptible-exposed-infected-removed (SEIR) model. However, training these hybrid networks is often unstable due to competing optimization objectives. As established in recent literature on ``gradient pathology," the gradient vectors derived from the data loss and the physical residual often point in conflicting directions, leading to slow convergence or optimization deadlock. While existing methods attempt to resolve this by balancing gradient magnitudes or projecting conflicting vectors, we propose a novel method, conflict-gated gradient scaling (CGGS), to address gradient conflicts in physics-informed neural networks for epidemiological modelling, ensuring stable and efficient training and a computationally efficient alternative. This method utilizes the cosine similarity between the data and physics gradients to dynamically modulate the penalty weight. Unlike standard annealing schemes that only normalize scales, CGGS acts as a geometric gate: it suppresses the physical constraint when directional conflict is high, allowing the optimizer to prioritize data fidelity, and restores the constraint when gradients align. We prove that this gating mechanism preserves the standard $O(1/T)$ convergence rate for smooth non-convex objectives, a guarantee that fails under fixed-weight or magnitude-balanced training when gradients conflict. We demonstrate that this mechanism autonomously induces a curriculum learning effect, improving parameter estimation in stiff epidemiological systems compared to magnitude-based baselines. Our empirical results show improved peak recovery and convergence over magnitude-based methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。