arXiv:2505.12027cs.LG2025-05被引 2

用关系令牌构建图模型基础框架,提升跨任务泛化能力

Relation-Aware Graph Foundation Model

  • 以关系令牌为基本单元,构建图神经网络的可扩展表示
  • 通过超网络自适应生成聚合器与分类器参数,适配多样关系
  • 适合需要跨数据集泛化的图学习研究者使用

近年来,大语言模型在自然语言处理任务中展现出卓越的泛化能力。类似地,图基础模型(GFMs)作为图学习的新方向,旨在通过大规模预训练实现跨多种数据集的泛化。然而,与依赖显式分词表示的语言模型不同,图结构缺乏统一的泛化单位,导致有效预训练策略设计困难。本文提出REEF框架,利用关系令牌作为GFMs的基本单元。受语言模型词表启发,构建关系词表以存储图中的关系信息。为适应多样化关系,引入两个超网络,根据关系令牌自适应生成图神经网络的聚合器和分类器参数。此外,设计另一个超网络用于构建数据集特异性投影器,并在初始节点表示中加入数据集级特征偏置,增强同一关系在不同数据集间的灵活性。同时采用图数据增强与混合数据集预训练策略,使REEF更有效地捕捉关系多样性,展现强大的泛化能力。大量实验表明,REEF在预训练与迁移学习任务上显著优于现有方法,证明其作为图应用基础模型的巨大潜力。

原文摘要 · Abstract (English)

In recent years, large language models (LLMs) have demonstrated remarkable generalization capabilities across various natural language processing (NLP) tasks. Similarly, graph foundation models (GFMs) have emerged as a promising direction in graph learning, aiming to generalize across diverse datasets through large-scale pre-training. However, unlike language models that rely on explicit token representations, graphs lack a well-defined unit for generalization, making it challenging to design effective pre-training strategies. In this work, we propose REEF, a novel framework that leverages relation tokens as the basic units for GFMs. Inspired by the token vocabulary in LLMs, we construct a relation vocabulary of relation tokens to store relational information within graphs. To accommodate diverse relations, we introduce two hypernetworks that adaptively generate the parameters of aggregators and classifiers in graph neural networks based on relation tokens. In addition, we design another hypernetwork to construct dataset-specific projectors and incorporate a dataset-level feature bias into the initial node representations, enhancing flexibility across different datasets with the same relation. Further, we adopt graph data augmentation and a mixed-dataset pre-training strategy, allowing REEF to capture relational diversity more effectively and exhibit strong generalization capabilities. Extensive experiments show that REEF significantly outperforms existing methods on both pre-training and transfer learning tasks, underscoring its potential as a powerful foundation model for graph-based applications.

图神经网络基础模型关系建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。