用正交贪心算法提升浅层神经网络对线性算子的学习效率
Orthogonal greedy algorithm for linear operator learning with shallow neural network
- 提出新型正交贪心算法，基于半内积空间优化核函数估计
- 在多个任务中实现逼近精度数量级提升，优于现有基线模型
- 理论证明算法最优逼近率，适用于偏微分方程与算子学习场景
贪心算法，特别是正交贪心算法（OGA），在浅层神经网络拟合函数和求解偏微分方程（PDEs）方面已被证明有效。本文将OGA扩展至线性算子学习任务，该任务等价于通过积分变换学习核函数。首先，我们提出一种新贪心算法，用于在新的半内积空间中进行核函数估计，可从数据中近似线性PDE的格林函数。其次，引入点态核估计的OGA，进一步提升逼近精度，在多种任务和基线模型中均实现阶数级的准确率改进。此外，我们对核函数估计问题进行了理论分析，建立了两种算法的最优逼近率，证实其有效性及在PDE与算子学习中的应用潜力。
原文摘要 · Abstract (English)
Greedy algorithms, particularly the orthogonal greedy algorithm (OGA), have proven effective in training shallow neural networks for fitting functions and solving partial differential equations (PDEs). In this paper, we extend the application of OGA to the tasks of linear operator learning, which is equivalent to learning the kernel function through integral transforms. Firstly, a novel greedy algorithm is developed for kernel estimation rate in a new semi-inner product, which can be utilized to approximate the Green's function of linear PDEs from data. Secondly, we introduce the OGA for point-wise kernel estimation to further improve the approximation rate, achieving orders of accuracy improvement across various tasks and baseline models. In addition, we provide a theoretical analysis on the kernel estimation problem and the optimal approximation rates for both algorithms, establishing their efficacy and potential for future applications in PDEs and operator learning tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。