提出不对称学习方法,解决谱图神经网络优化难题
Asymmetric Learning for Spectral Graph Neural Networks
- 区分图卷积与特征变换参数,动态调整梯度以改善优化
- 在18个数据集上提升谱图神经网络性能,异质图效果更显著
- 理论证明可降低海森矩阵块条件数,适合复杂图结构优化
谱图神经网络(GNN)的优化仍是关键挑战,但其内在机制尚不明确。本文研究谱图神经网络中图卷积参数与特征变换参数的本质差异及其对优化景观的影响,发现这些差异导致问题病态性,从而影响性能。为此,我们提出海森矩阵的块条件数概念,用于表征谱图神经网络优化中的病态程度,并设计了不对称学习方法,在训练中动态预处理梯度以缓解病态问题。理论上,该方法可降低块条件数,使优化更易进行。在18个基准数据集上的大量实验表明,不对称学习能持续提升谱图神经网络性能,尤其在异质图上表现更优,因异质图优化本就更复杂。代码已开源:https://github.com/Mia-321/asym-opt.git。
原文摘要 · Abstract (English)
Optimizing spectral graph neural networks (GNNs) remains a critical challenge in the field, yet the underlying processes are not well understood. In this paper, we investigate the inherent differences between graph convolution parameters and feature transformation parameters in spectral GNNs and their impact on the optimization landscape. Our analysis reveals that these differences contribute to a poorly conditioned problem, resulting in suboptimal performance. To address this issue, we introduce the concept of the block condition number of the Hessian matrix, which characterizes the difficulty of poorly conditioned problems in spectral GNN optimization. We then propose an asymmetric learning approach, dynamically preconditioning gradients during training to alleviate poorly conditioned problems. Theoretically, we demonstrate that asymmetric learning can reduce block condition numbers, facilitating easier optimization. Extensive experiments on eighteen benchmark datasets show that asymmetric learning consistently improves the performance of spectral GNNs for both heterophilic and homophilic graphs. This improvement is especially notable for heterophilic graphs, where the optimization process is generally more complex than for homophilic graphs. Code is available at https://github.com/Mia-321/asym-opt.git.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。