提出混合最小二乘/梯度下降法，加速MIONet训练
Hybrid Least Squares/Gradient Descent Methods for MIONets
- 用交替最小二乘优化各分支参数，提升训练效率
- 通过克罗内克积分解大矩阵，降低计算复杂度
- 支持带正则项的通用L2损失，适用于多种线性算子
本文提出一种高效的混合最小二乘/梯度下降（LSGD）方法，用于加速MIONet的训练。该方法推广了DeepONet的LSGD方法。由于MIONet是多个分支网络与主干网络逐元素乘积之和，可视为对各分支网络最后一层参数的多线性函数。因此，可通过交替最小二乘法分别优化各组参数，每次求解单个分支的最小二乘系统。为处理大规模系统矩阵，引入克罗内克积、Khatri-Rao积及张量置换矩阵，将大矩阵分解为多个小矩阵。该方法兼容带有正则化项的通用L²损失，且每项损失中可对MIONet输出施加线性算子。
原文摘要 · Abstract (English)
In this paper, we propose an efficient hybrid least squares/gradient descent (LSGD) method for MIONets to accelerate training. This method generalizes the LSGD method for DeepONets. Since MIONet is the sum of the entrywise product of multiple branch networks and a trunk network, it can be viewed as a multilinear function with respect to the last layer parameters of each branch network. These sets of parameters can be optimized using the alternating least squares method, where we solve the LS system for a single branch network in turn. To handle the large-sized system matrix, we introduce Kronecker and Khatri-Rao products and tensor permutation matrices to factor the large matrix into small ones. Our method is compatible with a general type of $L^2$ loss with regularization terms for the last layer parameters of each branch, where linear operators can be applied to the MIONet output in each loss term.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。