提出HRGrad方法,解决多尺度动力学中因参数变化导致的梯度冲突问题。
Conflict-Aware Harmonized Rotational Gradient for Multiscale Kinetic Regimes

- 通过隐式编码小参数实现任务序列化训练,缓解多任务学习中的梯度冲突。
- 引入梯度对齐度量,确保各任务损失更新方向一致,动态调节梯度强度。
- 在不同马赫数范围的BGK和线性输运方程上验证,显著提升APNN稳定性。
本文提出一种名为HRGrad的统一旋转梯度方法,用于同时求解具有不同小参数的多尺度时变动力学问题。这些参数在微观与宏观物理之间呈现渐近过渡,使得在所有范围内同时求解成为极具挑战性的多任务问题。不同渐近区域的任务求解常遭遇梯度冲突,导致多任务学习失败。为此,我们显式编码这些参数的隐含表示,使对应求解任务实现序列化训练。为缓解梯度冲突,我们对预测结果进行分段以构建任务损失,并引入一种新型梯度对齐度量,确保最终更新方向与每个任务损失梯度的点积为正。该度量维持所有任务损失的一致优化速率,并根据冲突程度动态调整梯度大小。此外,我们提供了HRGrad方法的收敛性数学证明,其在多种复杂渐近保持神经网络(APNN)场景下进行了评估。实验涵盖从低到高全范围克努森数的玻尔兹曼-格罗斯-克鲁克(BGK)方程与线性输运方程。结果表明,HRGrad能有效克服这些任务中APNN的“失效模式”。
原文摘要 · Abstract (English)
In this paper, we propose a harmonized rotational gradient method, termed HRGrad, for simultaneously tackling multiscale time-dependent kinetic problems with varying small parameters. These parameters exhibit asymptotic transitions from microscopic to macroscopic physics, making it a challenging multi-task problem to solve over all ranges simultaneously. Solving tasks in different asymptotic regions often encounter gradient conflicts, which can lead to the failure of multi-task learning. To address this challenge, we explicitly encode a hidden representation of these parameters, ensuring that the corresponding solving tasks are serialized for simultaneous training. Furthermore, to mitigate gradient conflicts, we segment the prediction results to construct task losses and introduce a novel gradient alignment metric to ensure a positive dot product between the final update and each loss-specific gradient. This metric maintains consistent optimization rates for all task losses and dynamically adjusts gradient magnitudes based on conflict levels. Moreover, we provide a mathematical proof demonstrating the convergence of the HRGrad method, which is evaluated across a range of challenging asymptotic-preserving neural networks (APNNs) scenarios. We conduct an extensive set of experiments encompassing the Bhatnagar-Gross-Krook (BGK) equation and the linear transport equation in all ranges of Knudsen number. Our results indicate that HRGrad effectively overcomes the `failure modes' of APNNs in these problems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。