针对大规模异构图,提出双感知预训练框架提升模型迁移能力。
Enhanced Pre-training of Graph Neural Networks for Million-Scale Heterogeneous Graphs
- 设计结构感知与语义感知双重预训练任务,兼顾图结构与语义信息。
- 在真实异构图上实现优于主流方法的下游性能,显著提升小样本场景下的泛化能力。
- 适合处理海量异构图数据的科研与工业场景,尤其适用于标注稀缺的任务。
近年来,图神经网络(GNN)推动了图数据挖掘的发展。然而,训练 GNN 需要大量有标签的任务特定数据,这既昂贵又难以获取。为减少对标注数据的依赖,现有研究提出自监督方式预训练 GNN,再用于少量标注数据的下游任务。但多数方法仅针对同构图设计,未考虑原始数据与理想可迁移语义数据间的语义差异。本文提出一种高效框架,用于大规模异构图上的 GNN 预训练。首先设计结构感知预训练任务,捕捉异构图中的结构特性;随后设计语义感知任务,解决语义不匹配问题。具体地,构建由语义邻居组成的扰动子空间,使模型更关注语义空间中的通用知识,从而学习更具迁移性的特征。在真实世界大规模异构图上进行广泛实验,验证所提方法优于当前最优基线。代码已开源:https://github.com/sunshy-1/PHE。
原文摘要 · Abstract (English)
In recent years, graph neural networks (GNNs) have facilitated the development of graph data mining. However, training GNNs requires sufficient labeled task-specific data, which is expensive and sometimes unavailable. To be less dependent on labeled data, recent studies propose to pre-train GNNs in a self-supervised manner and then apply the pre-trained GNNs to downstream tasks with limited labeled data. However, most existing methods are designed solely for homogeneous graphs (real-world graphs are mostly heterogeneous) and do not consider semantic mismatch (the semantic difference between the original data and the ideal data containing more transferable semantic information). In this paper, we propose an effective framework to pre-train GNNs on the large-scale heterogeneous graph. We first design a structure-aware pre-training task, which aims to capture structural properties in heterogeneous graphs. Then, we design a semantic-aware pre-training task to tackle the mismatch. Specifically, we construct a perturbation subspace composed of semantic neighbors to help deal with the semantic mismatch. Semantic neighbors make the model focus more on the general knowledge in the semantic space, which in turn assists the model in learning knowledge with better transferability. Finally, extensive experiments are conducted on real-world large-scale heterogeneous graphs to demonstrate the superiority of the proposed method over state-of-the-art baselines. Code available at https://github.com/sunshy-1/PHE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。