用最小二乘+梯度下降加速DeepONet训练，提升效率。
Hybrid Least Squares/Gradient Descent Methods for DeepONets
- 将分支网络参数用最小二乘法求解，其余参数用梯度下降更新。
- 分解大线性系统为分支与主干网络两个小问题分别求解。
- 适用于带正则化的L²损失，支持物理信息监督学习。
我们提出一种高效的混合最小二乘/梯度下降方法，以加速DeepONet的训练。由于DeepONet的输出对分支网络最后一层参数呈线性关系，这些参数可通过最小二乘（LS）求解，其余隐藏层参数则通过梯度下降更新。然而，构建所有分支与主干输入组合的完整LS系统会产生一个规模过大、难以直接求解的线性问题。为此，我们的方法将大型LS系统分解为两个更小、可管理的子问题——一个针对分支网络，另一个针对主干网络，并分别求解。该方法进一步推广至包含最后一层参数正则化项的广义L²损失，涵盖无监督学习中的物理信息损失情形。
原文摘要 · Abstract (English)
We propose an efficient hybrid least squares/gradient descent method to accelerate DeepONet training. Since the output of DeepONet can be viewed as linear with respect to the last layer parameters of the branch network, these parameters can be optimized using a least squares (LS) solve, and the remaining hidden layer parameters are updated by means of gradient descent form. However, building the LS system for all possible combinations of branch and trunk inputs yields a prohibitively large linear problem that is infeasible to solve directly. To address this issue, our method decomposes the large LS system into two smaller, more manageable subproblems $\unicode{x2014}$ one for the branch network and one for the trunk network $\unicode{x2014}$ and solves them separately. This method is generalized to a broader type of $L^2$ loss with a regularization term for the last layer parameters, including the case of unsupervised learning with physics-informed loss.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。