证明线性GNN梯度训练能以指数速度收敛到全局最优。
Convergence of gradient based training for linear Graph Neural Networks
- 通过梯度流分析线性GNN的训练过程
- 收敛速率依赖初始权重与图移位算子，实证验证
- 适用于研究GNN理论性质的学者
图神经网络（GNN）在分子生物学和社交网络等图结构学习任务中表现优异，但其经验性能的理论基础仍不清晰。本文研究线性GNN在梯度流下的收敛性，证明使用均方损失时，线性GNN的梯度流训练能以指数速率收敛至全局最小值。该收敛速率显式依赖于初始权重和图移位算子，已在经典图模型生成的合成数据集及真实世界数据集上验证。此外，还讨论了在全局最小值处使总权重最小化的梯度流。除梯度流外，也研究了梯度下降训练下的收敛性，该方法被视为梯度流的离散化。
原文摘要 · Abstract (English)
Graph Neural Networks (GNNs) are powerful tools for addressing learning problems on graph structures, with a wide range of applications in molecular biology and social networks. However, the theoretical foundations underlying their empirical performance are not well understood. In this article, we examine the convergence of gradient dynamics in the training of linear GNNs. Specifically, we prove that the gradient flow training of a linear GNN with mean squared loss converges to the global minimum at an exponential rate. The convergence rate depends explicitly on the initial weights and the graph shift operator, which we validate on synthetic datasets from well-known graph models and real-world datasets. Furthermore, we discuss the gradient flow that minimizes the total weights at the global minimum. In addition to the gradient flow, we study the convergence of linear GNNs under gradient descent training, an iterative scheme viewed as a discretization of gradient flow.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。