提出高效计算物理损失中NTK权重的方法，保证收敛且大幅降低计算开销。
Convergence and Sketching-Based Efficient Computation of Neural Tangent Kernel Weights in Physics-Based Loss
- 基于随机化矩阵分解与预测-校正框架，实现无偏的NTK估计
- 理论证明自适应NTK权重在特定条件下可保证梯度下降收敛
- 适合需要多目标优化的物理信息神经网络研究者使用
在多目标优化中，多个损失项通过加权求和形成单一目标函数，权重需根据某种元目标合理平衡。例如，在物理信息神经网络（PINNs）中，常采用自适应权重以提升模型泛化能力。一种流行选择是基于神经正切核（NTK）的权重，它描述了训练过程中网络在预测空间中的演化行为。然而，此类自适应权重算法的收敛性尚不明确；且在训练过程中频繁更新这些权重会进一步增加计算负担。本文在适当条件下证明了：引入自适应NTK权重的梯度下降方法在某种意义下具有收敛性。随后，我们提出一种受预测-校正方法和矩阵压缩启发的随机算法，能以任意小的离散误差产生对NTK的无偏估计。最后，通过数值实验验证了理论结果，并展示了该随机算法的有效性。
原文摘要 · Abstract (English)
In multi-objective optimization, multiple loss terms are weighted and added together to form a single objective. These weights are chosen to properly balance the competing losses according to some meta-goal. For example, in physics-informed neural networks (PINNs), these weights are often adaptively chosen to improve the network's generalization error. A popular choice of adaptive weights is based on the neural tangent kernel (NTK) of the PINN, which describes the evolution of the network in predictor space during training. The convergence of such an adaptive weighting algorithm is not clear a priori. Moreover, these NTK-based weights would be updated frequently during training, further increasing the computational burden of the learning process. In this paper, we prove that under appropriate conditions, gradient descent enhanced with adaptive NTK-based weights is convergent in a suitable sense. We then address the problem of computational efficiency by developing a randomized algorithm inspired by a predictor-corrector approach and matrix sketching, which produces unbiased estimates of the NTK up to an arbitrarily small discretization error. Finally, we provide numerical experiments to support our theoretical findings and to show the efficacy of our randomized algorithm. Code Availability: https://github.com/maxhirsch/Efficient-NTK
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。