arXiv:2411.07672cs.LG2024-11被引 6

发现图神经网络的结构学习实际提升性能的关键是预训练表示,而非重构图结构。

Rethinking Structure Learning For Graph Neural Networks

  • 提出三步式结构学习框架,系统拆解图结构学习的各环节作用
  • 实验证明重构图结构后节点表征的互信息无提升,说明结构学习无效
  • 强调预训练表征和自训练才是性能关键,适合关注GNN设计本质的研究者

为提升图神经网络(GNN)性能,图结构学习(GSL)被广泛用于重构或优化原始图结构,以应对异质性、过挤压及噪声等问题。尽管普遍认为GSL能提升性能,但其常导致训练时间延长和超参数调优复杂。此外,现有方法间差异模糊,缺乏理论分析。近期研究指出,在公平对比下,GSL并不总优于基线模型。为此,本文提出新框架,包含三步:构建GSL基础表示、生成新结构、视图融合,以理解其有效性。在图卷积后,分析原拓扑与新拓扑下节点表示的互信息(MI),发现无论采用何种结构构造方法,新结构下的表征并未带来额外互信息增益。通过消融实验确认,真正提升性能的是预训练的GSL基础表示,而多数情况下结构学习无法改善性能。该发现促使我们重新思考GNN设计中的核心组件,如自训练和结构编码,而非依赖结构学习。

原文摘要 · Abstract (English)

To improve the performance of Graph Neural Networks (GNNs), Graph Structure Learning (GSL) has been extensively applied to reconstruct or refine original graph structures, effectively addressing issues like heterophily, over-squashing, and noisy structures. While GSL is generally thought to improve GNN performance, it often leads to longer training times and more hyperparameter tuning. Besides, the distinctions among current GSL methods remain ambiguous from the perspective of GNN training, and there is a lack of theoretical analysis to quantify their effectiveness. Recent studies further suggest that, under fair comparisons with the same hyperparameter tuning, GSL does not consistently outperform baseline GNNs. This motivates us to ask a critical question: is GSL really useful for GNNs? To address this question, this paper makes two key contributions. First, we propose a new GSL framework, which includes three steps: GSL base (the representation used for GSL) construction, new structure construction, and view fusion, to better understand the effectiveness of GSL in GNNs. Second, after graph convolution, we analyze the differences in mutual information (MI) between node representations derived from the original topology and those from the newly constructed topology. Surprisingly, our empirical observations and theoretical analysis show that no matter which type of graph structure construction methods are used, after feeding the same GSL bases to the newly constructed graph, there is no MI gain compared to the original GSL bases. To fairly reassess the effectiveness of GSL, we conduct ablation experiments and find that it is the pretrained GSL bases that enhance GNN performance, and in most cases, GSL cannot improve GNN performance. This finding encourages us to rethink the essential components in GNNs, such as self-training and structural encoding, in GNN design rather than GSL.

图神经网络结构学习自训练表征学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。