让MLP学会图结构信息,推理速度更快且不依赖邻居数据。
Training MLPs on Graphs without Supervision
- 通过自监督学习对齐GNN与MLP的表示,融合图结构特征。
- 在20个数据集上超越现有方法,尤其在未见节点场景表现优异。
- 适合需要快速推理且结构信息关键的应用,如金融反欺诈。
图神经网络(GNN)在各类图学习任务中表现优异,但其推理时依赖邻域聚合,难以满足实时金融反欺诈等低延迟场景需求。为此,近年研究尝试将教师GNN的知识蒸馏到基于节点内容训练的学生MLP中以加速推理。然而,这些方法在推断未知节点时对结构信息利用不足。为此,我们提出SimMLP,一种图上MLP的自监督学习框架,旨在将丰富的结构信息完全融入MLP。SimMLP是首个能在最优情况下实现与GNN等效的MLP学习方法。核心思想是通过自监督学习,使图上下文感知的GNN与无邻域依赖的MLP表示对齐,从而充分整合结构信息。我们提供全面的理论分析,基于互信息和归纳偏置证明了SimMLP与GNN的等价性,凸显其强大的结构学习能力。此外,在20个基准数据集上进行的大量实验表明,无论在节点分类、链接预测还是图分类任务中,SimMLP均优于当前最佳基线,尤其是在涉及未见节点的归纳与冷启动节点分类场景中表现突出。代码已公开于:https://github.com/Zehong-Wang/SimMLP。
原文摘要 · Abstract (English)
Graph Neural Networks (GNNs) have demonstrated their effectiveness in various graph learning tasks, yet their reliance on neighborhood aggregation during inference poses challenges for deployment in latency-sensitive applications, such as real-time financial fraud detection. To address this limitation, recent studies have proposed distilling knowledge from teacher GNNs into student Multi-Layer Perceptrons (MLPs) trained on node content, aiming to accelerate inference. However, these approaches often inadequately explore structural information when inferring unseen nodes. To this end, we introduce SimMLP, a Self-supervised framework for learning MLPs on graphs, designed to fully integrate rich structural information into MLPs. Notably, SimMLP is the first MLP-learning method that can achieve equivalence to GNNs in the optimal case. The key idea is to employ self-supervised learning to align the representations encoded by graph context-aware GNNs and neighborhood dependency-free MLPs, thereby fully integrating the structural information into MLPs. We provide a comprehensive theoretical analysis, demonstrating the equivalence between SimMLP and GNNs based on mutual information and inductive bias, highlighting SimMLP's advanced structural learning capabilities. Additionally, we conduct extensive experiments on 20 benchmark datasets, covering node classification, link prediction, and graph classification, to showcase SimMLP's superiority over state-of-the-art baselines, particularly in scenarios involving unseen nodes (e.g., inductive and cold-start node classification) where structural insights are crucial. Our codes are available at: https://github.com/Zehong-Wang/SimMLP.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。