arXiv:2606.29773cs.LG2026-06

将图神经网络与大语言模型联合预训练,提升图级任务性能。

GLIP: Graph and LLM Joint Pretraining for Graph-Level Tasks

论文配图:GLIP: Graph and LLM Joint Pretraining for Graph-Level Tasks
图 1 · 摘自论文原文
  • 设计多令牌选择策略,捕捉结构与特征的双重信息
  • 通过扩散投影增强上下文,实现全局与局部信号融合
  • 结合语义判断与对比损失,统一层次监督机制

图广泛用于建模关系系统,应用涵盖社交网络、金融和生物医学等领域。图神经网络(GNN)已成为学习图表示的主流方法。随着大语言模型(LLM)的发展,近期研究尝试将GNN与LLM结合,但多数工作集中于节点级和边级任务,而需要捕捉更复杂结构与特征信息的图级任务仍相对未被充分探索。此外,图预训练是缓解标签稀缺问题的常用策略,但现有方法如GraphCL仅针对GNN设计,未涉及LLM。为此,我们提出GLIP:一种面向图级任务的图-语言模型联合预训练框架。GLIP首先通过图增强构建正负样本对,并引入多令牌选择策略识别在结构和特征上均具信息量的片段。进一步利用基于扩散的投影器,为这些片段注入上下文信息,使模型能从全局与局部视角捕获信号。最后,采用联合目标函数,整合LLM的语义判断与对比对齐损失,确保在语义与结构层面的一致性监督。预训练后,使用少量标注数据微调,大量实验表明其在图级分类与推理任务中优于现有最先进方法。源代码已公开于https://anonymous.4open.science/r/GLIP。

原文摘要 · Abstract (English)

Graphs are widely used to model relational systems, with applications in domains such as social networks, finance, and biomedicine. Graph neural networks (GNNs) have become a mainstream approach for learning graph representations. With the rise of large language models (LLMs), recent studies have attempted to combine GNNs with LLMs. However, most existing works concentrate on node-level and edge-level tasks, while graph-level tasks, which require capturing more complex structural and feature information, remain relatively underexplored. Moreover, graph pretraining is a widely adopted strategy to alleviate the challenge of label scarcity. Most existing approaches are designed solely for GNNs such as GraphCL, leaving LLMs uninvolved in the process. To address these limitations, we propose GLIP, a Graph-LLM JoInt Pretraining framework for graph-level tasks. GLIP first performs graph augmentation to construct positive and negative pairs and introduces a multi-token selection strategy to identify patches informative in both structure and features. It further leverages a diffusion-based projector to enrich them with contextual information, enabling GLIP to capture signals from both global and local perspectives. Finally, GLIP employs a joint objective that integrates the LLM's semantic judgments with a contrastive alignment loss, ensuring consistent supervision at both the semantic and structural levels. After pretraining, GLIP is fine-tuned with limited labeled data for downstream tasks, and extensive experiments show that it outperforms state-of-the-art methods on graph-level classification and reasoning tasks. Our source code is publicly available at https://anonymous.4open.science/r/GLIP.

图神经网络大模型联合预训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。