让大模型理解图结构,用文本词汇统一表示节点,提升跨图泛化能力。
LLM as GNN: Graph Vocabulary Learning for Text-Attributed Graph Foundation Models
- 用提示词让LLM模拟GNN流程,实现图文无缝融合
- 构建语言式图词汇表,避免外生词爆炸,支持任务模板复用
- 适用于多领域文本图任务,尤其适合需要跨图迁移的场景
文本属性图(TAGs)在现实场景中普遍存在,每个节点关联文本描述,具有独特结构与领域知识。现有方法将大语言模型(LLM)与图神经网络(GNN)结合时,常采用两阶段解耦架构,难以发挥协同效应。更严重的是,传统方法为图节点分配大量未登录词(OOV),导致语义专属、词汇膨胀,且无法适配任务导向的提示模板,限制了跨图和跨任务的可迁移性。为此,我们提出PromptGFM,一种基于图词汇学习的文本属性图基础模型。该模型包含两个核心组件:(1)图理解模块,通过显式提示使LLM在文本空间内复现最精细的GNN工作流,实现无缝的GNN-LLM融合与优雅的图-文对齐;(2)图推理模块,建立基于语言的图词汇表,保障表达力、可迁移性与可扩展性,支持可读的指令用于LLM微调。大量实验表明,PromptGFM在多样图与任务上均展现出优越性能与强泛化能力。代码已开源:https://github.com/agiresearch/PromptGFM。
原文摘要 · Abstract (English)
Text-Attributed Graphs (TAGs), where each node is associated with text descriptions, are ubiquitous in real-world scenarios. They typically exhibit distinctive structure and domain-specific knowledge, motivating the development of a Graph Foundation Model (GFM) that generalizes across diverse graphs and tasks. Despite large efforts to integrate Large Language Models (LLMs) and Graph Neural Networks (GNNs) for TAGs, existing approaches suffer from decoupled architectures with two-stage alignment, limiting their synergistic potential. Even worse, existing methods assign out-of-vocabulary (OOV) tokens to graph nodes, leading to graph-specific semantics, token explosion, and incompatibility with task-oriented prompt templates, which hinders cross-graph and cross-task transferability. To address these challenges, we propose PromptGFM, a versatile GFM for TAGs grounded in graph vocabulary learning. PromptGFM comprises two key components: (1) Graph Understanding Module, which explicitly prompts LLMs to replicate the finest GNN workflow within the text space, facilitating seamless GNN-LLM integration and elegant graph-text alignment; (2) Graph Inference Module, which establishes a language-based graph vocabulary ensuring expressiveness, transferability, and scalability, enabling readable instructions for LLM fine-tuning. Extensive experiments demonstrate our superiority and transferability across diverse graphs and tasks. The code is available at this: https://github.com/agiresearch/PromptGFM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。