用稀疏索博列夫范数提升图神经网络效率,兼顾高阶关系建模。
Higher-Order GNNs Meet Efficiency: Sparse Sobolev Graph Neural Networks
- 基于拉普拉斯矩阵的哈达玛幂构造稀疏卷积算子,保持图结构稀疏性。
- 在多个节点分类任务中达到领先性能,运行速度优于主流GNN模型。
- 理论证明模型对图扰动具有鲁棒性,适合大规模图数据应用。
图神经网络(GNN)在建模图中节点间关系方面表现优异,但捕捉高阶邻域关系仍是大规模网络的挑战。现有方法多通过图拉普拉斯或邻接矩阵的幂来利用高阶邻居信息,但带来计算与内存开销增加的问题。我们基于图谱理论发现:拉普拉斯矩阵的正规幂与哈达玛幂在谱上行为相似,这一观察为高效捕获高阶信息提供了新思路。为此,提出基于图信号稀疏索博列夫范数的新卷积算子——稀疏索博列夫图神经网络(S2-GNN)。S2-GNN通过矩阵间的哈达玛乘积维持表示稀疏性,并采用逐级增长的哈达玛幂滤波器生成多样化函数。理论上分析了模型对图扰动的稳定性。在多种图挖掘、半监督节点分类及计算机视觉任务中进行全面评估,结果显示其在性能与运行时间上均优于现有先进GNN模型。
原文摘要 · Abstract (English)
Graph Neural Networks (GNNs) have shown great promise in modeling relationships between nodes in a graph, but capturing higher-order relationships remains a challenge for large-scale networks. Previous studies have primarily attempted to utilize the information from higher-order neighbors in the graph, involving the incorporation of powers of the shift operator, such as the graph Laplacian or adjacency matrix. This approach comes with a trade-off in terms of increased computational and memory demands. Relying on graph spectral theory, we make a fundamental observation: the regular and the Hadamard power of the Laplacian matrix behave similarly in the spectrum. This observation has significant implications for capturing higher-order information in GNNs for various tasks such as node classification and semi-supervised learning. Consequently, we propose a novel graph convolutional operator based on the sparse Sobolev norm of graph signals. Our approach, known as Sparse Sobolev GNN (S2-GNN), employs Hadamard products between matrices to maintain the sparsity level in graph representations. S2-GNN utilizes a cascade of filters with increasing Hadamard powers to generate a diverse set of functions. We theoretically analyze the stability of S2-GNN to show the robustness of the model against possible graph perturbations. We also conduct a comprehensive evaluation of S2-GNN across various graph mining, semi-supervised node classification, and computer vision tasks. In particular use cases, our algorithm demonstrates competitive performance compared to state-of-the-art GNNs in terms of performance and running time.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。