让MLP学到图中最关键节点的信息,提升推理速度与精度。
InfGraND: An Influence-Guided GNN-to-MLP Knowledge Distillation
- 根据节点对图结构的重要性动态选择知识蒸馏目标
- 在7个图数据集上显著超越现有方法,性能更优
- 适合低延迟、资源受限场景的模型轻量化部署
图神经网络(GNN)是图数据分析的主流模型,但其依赖聚合与更新操作,在低延迟或资源受限场景下效率不足。简单多层感知机(MLP)虽计算高效,但监督训练常表现不佳。知识蒸馏(KD)通过将GNN教师的知识迁移至MLP学生可缓解此问题。然而,现有方法通常均匀传递知识或依赖预测不确定性等非图指标,忽视了根本问题:‘一个节点对图结构有多重要?’本文提出InfGraND——一种基于影响度的图到MLP知识蒸馏框架,通过识别并优先选择结构上关键的节点来引导蒸馏过程,确保学生模型学习最具代表性的图信息。此外,InfGraND通过一次性的多跳邻域特征预计算,将结构信息嵌入学生输入,避免推理时开销。在七个同质性图基准数据集上的横切与归纳设置评估中,InfGraND持续优于现有方法,证明其在真实低延迟应用中的实用性。
原文摘要 · Abstract (English)
Graph Neural Networks (GNNs) are the go-to model for graph data analysis. However, GNNs rely on two key operations - aggregation and update, which can pose challenges for low-latency inference tasks or resource-constrained scenarios. Simple Multi-Layer Perceptrons (MLPs) offer a computationally efficient alternative. Yet, training an MLP in a supervised setting often leads to suboptimal performance. Knowledge Distillation (KD) from a GNN teacher to an MLP student has emerged to bridge this gap. However, most KD methods either transfer knowledge uniformly across all nodes or rely on graph-agnostic indicators such as prediction uncertainty. We argue this overlooks a more fundamental, graph-centric inquiry: "How important is a node to the structure of the graph?" We introduce a framework, InfGraND, an Influence-guided Graph KNowledge Distillation from GNN to MLP that addresses this by identifying and prioritizing structurally influential nodes to guide the distillation process, ensuring that the MLP learns from the most critical parts of the graph. Additionally, InfGraND embeds structural awareness in MLPs through one-time multi-hop neighborhood feature pre-computation, which enriches the student MLP's input and thus avoids inference-time overhead. Our rigorous evaluation in transductive and inductive settings across seven homophilic graph benchmark datasets shows InfGraND consistently outperforms prior GNN to MLP KD methods, demonstrating its practicality for numerous latency-critical applications in real-world settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。