提出一种图谱稀疏化方法,让大型图神经网络更快更准地训练。
Large-Scale Spectral Graph Neural Networks via Laplacian Sparsification: Technical Report
- 用拉普拉斯矩阵稀疏化近似谱图网络传播模式
- 在Ogbn-papers100M等超大规模数据集上效果超越基线模型
- 支持端到端训练,适合处理原始文本特征的复杂任务
图神经网络在图表示学习中具有关键作用。其中,采用多项式滤波器的谱图神经网络在同质与异质图结构上均表现优异。然而,其在大规模图上的可扩展性受限,因需多次前向传播来学习多项式系数。现有方法通过移除输入节点特征的线性层以提升效率,但这会破坏端到端训练,影响性能,且不适用于高维特征。为此,我们提出「拉普拉斯稀疏化的谱图神经网络(SGNN-LS)」,一种新型图谱稀疏化方法,用于近似谱图网络的传播模式。理论证明该方法能有效逼近固定和可学习的多项式滤波器。该方法支持输入特征的线性层,实现端到端训练,并可处理原始文本特征。我们在涵盖多种图规模与特性的数据集上进行广泛实验,结果表明,本方法在效率与效果上均显著优于对应近似基线模型,尤其在Ogbn-papers100M(1.11亿节点,160亿边)和MAG-scholar-C(280万特征)上表现突出。
原文摘要 · Abstract (English)
Graph Neural Networks (GNNs) play a pivotal role in graph-based tasks for their proficiency in representation learning. Among the various GNN methods, spectral GNNs employing polynomial filters have shown promising performance on tasks involving both homophilous and heterophilous graph structures. However, The scalability of spectral GNNs on large graphs is limited because they learn the polynomial coefficients through multiple forward propagation executions during forward propagation. Existing works have attempted to scale up spectral GNNs by eliminating the linear layers on the input node features, a change that can disrupt end-to-end training, potentially impact performance, and become impractical with high-dimensional input features. To address the above challenges, we propose "Spectral Graph Neural Networks with Laplacian Sparsification (SGNN-LS)", a novel graph spectral sparsification method to approximate the propagation patterns of spectral GNNs. We prove that our proposed method generates Laplacian sparsifiers that can approximate both fixed and learnable polynomial filters with theoretical guarantees. Our method allows the application of linear layers on the input node features, enabling end-to-end training as well as the handling of raw text features. We conduct an extensive experimental analysis on datasets spanning various graph scales and properties to demonstrate the superior efficiency and effectiveness of our method. The results show that our method yields superior results in comparison with the corresponding approximated base models, especially on dataset Ogbn-papers100M(111M nodes, 1.6B edges) and MAG-scholar-C (2.8M features).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。