用MLP模拟图神经网络，提速379倍还更准
Teaching MLPs to Master Heterogeneous Graph-Structured Knowledge for Efficient and Accurate Inference
- 用教师图网络给学生MLP传知识，不依赖图结构
- 在6个数据集上性能超原图网络，大图快379倍
- 适合对速度要求高的实际部署场景
异构图神经网络（HGNN）在各类异构图学习任务中表现优异，能有效捕捉复杂关系与多样语义。但其依赖图结构带来的邻居获取延迟，难以满足低延迟应用需求。受近期GNN到MLP知识蒸馏框架启发，本文提出HG2M与HG2M+，将HGNN的高性能与MLP的高效推理结合。HG2M直接以节点特征为输入，以教师HGNN的软标签为目标训练学生MLP；HG2M+通过可靠节点蒸馏和可靠元路径蒸馏，进一步将可靠的异构语义知识注入学生模型。在六个异构图数据集上的实验表明，尽管无结构依赖，HG2M仍能达到甚至超过HGNN的性能，显著优于基础MLP。尤其在大规模IGB-3M-19数据集上，推理速度相较HGNN提升379.24倍，展现出对延迟敏感场景的强大适应能力。
原文摘要 · Abstract (English)
Heterogeneous Graph Neural Networks (HGNNs) have achieved promising results in various heterogeneous graph learning tasks, owing to their superiority in capturing the intricate relationships and diverse relational semantics inherent in heterogeneous graph structures. However, the neighborhood-fetching latency incurred by structure dependency in HGNNs makes it challenging to deploy for latency-constrained applications that require fast inference. Inspired by recent GNN-to-MLP knowledge distillation frameworks, we introduce HG2M and HG2M+ to combine both HGNN's superior performance and MLP's efficient inference. HG2M directly trains student MLPs with node features as input and soft labels from teacher HGNNs as targets, and HG2M+ further distills reliable and heterogeneous semantic knowledge into student MLPs through reliable node distillation and reliable meta-path distillation. Experiments conducted on six heterogeneous graph datasets show that despite lacking structural dependencies, HG2Ms can still achieve competitive or even better performance than HGNNs and significantly outperform vanilla MLPs. Moreover, HG2Ms demonstrate a 379.24$\times$ speedup in inference over HGNNs on the large-scale IGB-3M-19 dataset, showcasing their ability for latency-sensitive deployments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。