研究浅层图卷积网络在采样点云上的训练连续性,建立网格与采样独立的理论基础。
Manifold limit for the training of shallow graph convolutional neural networks
- 从泛函分析视角,将图信号视为流形上函数的离散化,定义跨分辨率训练一致性。
- 证明正则化经验风险最小化泛函Γ-收敛,全局极小值弱收敛且函数一致收敛。
- 适用于关注图神经网络理论性质、高维几何学习的研究者。
我们在流形假设下研究了浅层图卷积神经网络(GCNN)在采样点云的邻近图上的训练从离散到连续的保真性。图卷积通过图拉普拉斯算子的谱定义,其低频谱近似于底层光滑流形的拉普拉斯-贝尔特拉米算子。具有无限宽度的浅层GCNN可视为参数空间上测度的线性泛函。从这一泛函分析视角出发,图信号被视为流形上函数的空间离散化,从而导出跨图分辨率的自然训练数据一致性。为获得收敛结果,连续参数空间被选为单位球的弱紧乘积,并对输出权重和偏置施加Sobolev正则性,但不对卷积参数施加。对应的离散参数空间继承了相应的谱衰减,并额外受制于适应图拉普拉斯算子信息频段的频率截断。在此假设下,我们证明了正则化经验风险最小化泛函的Γ-收敛,以及其全局极小值在参数测度弱收敛和函数在紧集上一致收敛的意义下的收敛性。这为这类网络训练的网格与采样独立性提供了形式化依据。
原文摘要 · Abstract (English)
We study the discrete-to-continuum consistency of the training of shallow graph convolutional neural networks (GCNNs) on proximity graphs of sampled point clouds under a manifold assumption. Graph convolution is defined spectrally via the graph Laplacian, whose low-frequency spectrum approximates that of the Laplace-Beltrami operator of the underlying smooth manifold, and shallow GCNNs of possibly infinite width are linear functionals on the space of measures on the parameter space. From this functional-analytic perspective, graph signals are seen as spatial discretizations of functions on the manifold, which leads to a natural notion of training data consistent across graph resolutions. To enable convergence results, the continuum parameter space is chosen as a weakly compact product of unit balls, with Sobolev regularity imposed on the output weight and bias, but not on the convolutional parameter. The corresponding discrete parameter spaces inherit the corresponding spectral decay, and are additionally restricted by a frequency cutoff adapted to the informative spectral window of the graph Laplacians. Under these assumptions, we prove $Γ$-convergence of regularized empirical risk minimization functionals and corresponding convergence of their global minimizers, in the sense of weak convergence of the parameter measures and uniform convergence of the functions over compact sets. This provides a formalization of mesh and sample independence for the training of such networks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。