提出可学习图块,解决无文本图数据的特征异质性问题。
Handling Feature Heterogeneity with Learnable Graph Patches

- 将图分解为可学习的最小语义单元——图块,独立构建结构。
- 在多领域图上预训练,下游任务性能随预训练数据量增加而提升。
- 不依赖领域信息,适用于跨域图模型迁移,适合图神经网络研究者。
近年来,基础模型与图预训练技术的快速发展推动了通用预训练图模型(图基础模型,GFM)的构建。然而,现有模型难以处理无文本信息的图数据中的特征异质性,限制了图模型在不同数据集间的迁移能力。为此,我们提出可学习图块的概念,将其视为任意图数据的最小语义单元。通过展开节点特征并分别构建对应的图块结构,将图分解为可学习图块。设计框架从跨域图数据中挖掘可迁移信息:提取图块后,采用图块编码器提取每个单元的知识,再通过图块聚合器学习单元如何组合成整体。由于其领域无关特性,该模型可应用于不同领域的下游数据。我们还分析了方法与现有图模型的关系,以及生成节点嵌入的可迁移性。实验表明,该方法不仅具备使用多领域图进行预训练的能力,还在多种下游数据集和任务上表现更优,且随着预训练数据量增加,下游性能持续提升。
原文摘要 · Abstract (English)
In recent years, the rapid development of foundation models and graph pre-training technologies has spurred increasing interest in constructing a universal pre-trained graph model or Graph Foundation Model (GFM). However, a significant challenge is that existing models are unable to address feature heterogeneity in graph data without textual information, which hinders the transferability of graph models across different datasets. To bridge this gap, we propose the concept of learnable graph patches, which we regard as the smallest semantic units of any graph data. We decompose the graph into learnable graph patches by unfolding the node features and constructing corresponding patch structures separately. We then design a framework that mines transferable information from graph data across domains. Specifically, after extracting graph patches, we propose a patch encoder to extract knowledge from each unit and a patch aggregator to learn how the units are combined into a whole. Due to its domain-agnostic nature, the model can be applied to downstream data across different domains. Furthermore, we analyze the connection between our method and existing graph models, as well as the transferability of the node embeddings it generates. Empirically, our method not only achieves the capability to use multi-domain graphs for pre-training, but also shows enhanced performance across various downstream datasets and tasks. Moreover, we observe consistent improvement in downstream performance as the volume of pre-training data increases.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。