arXiv:2411.06070cs.LGcs.AI2024-11NeurIPS被引 97

用计算树作为通用词汇构建图模型,提升跨任务泛化能力。

GFT: Graph Foundation Model with Transferable Tree Vocabulary

  • 将消息传递过程生成的计算树视为可迁移的词汇单元
  • 在多个图学习任务上实现优于现有方法的泛化性能
  • 适合研究图神经网络通用建模与跨领域应用的学者

受ChatGPT等基础模型成功的启发,随着图数据的广泛应用,图基础模型(GFMs)有望在科研、社交网络分析、药物发现和电商等领域带来深远影响。尽管预训练图神经网络取得显著进展,但尚无能有效支持多种图学习任务的图基础模型。构建图基础模型的关键在于定义可在不同任务和领域间共享的可迁移模式。与图像和文本不同,图数据的可迁移模式仍是一个开放问题。本文提出将图上的可迁移模式重新理解为计算树——即由消息传递过程衍生出的树结构。基于此,我们构建了名为GFT(Graph Foundation Model with Transferable Tree Vocabulary)的跨任务、跨域图基础模型。通过将计算树视为可迁移词汇中的标记,GFT提升了模型泛化能力并降低了负迁移风险。理论分析与大量实验证明了计算树的可迁移性,并验证了GFT在多样图学习任务与领域中的有效性。开源代码与数据已发布于https://github.com/Zehong-Wang/GFT。

原文摘要 · Abstract (English)

Inspired by the success of foundation models in applications such as ChatGPT, as graph data has been ubiquitous, one can envision the far-reaching impacts that can be brought by Graph Foundation Models (GFMs) with broader applications in the areas such as scientific research, social network analysis, drug discovery, and e-commerce. Despite the significant progress of pre-trained graph neural networks, there haven't been GFMs that can achieve desired performance on various graph-learning-related tasks. Building GFMs may rely on a vocabulary that encodes transferable patterns shared among different tasks and domains. Unlike image and text, defining such transferable patterns for graphs remains an open question. In this paper, we aim to bridge this gap by rethinking the transferable patterns on graphs as computation trees -- i.e., tree structures derived from the message-passing process. Based on this insight, we propose a cross-task, cross-domain graph foundation model named GFT, short for Graph Foundation model with transferable Tree vocabulary. By treating computation trees as tokens within the transferable vocabulary, GFT improves model generalization and reduces the risk of negative transfer. The theoretical analyses and extensive experimental studies have demonstrated the transferability of computation trees and shown the effectiveness of GFT across diverse tasks and domains in graph learning. The open source code and data are available at https://github.com/Zehong-Wang/GFT.

图神经网络基础模型计算树迁移学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。