arXiv:2501.11478cs.CLcs.AI2025-01ACL被引 16

将图结构视为新语言,用LLM更高效地学习图数据。

Each Graph is a New Language: Graph Learning with LLMs

  • 把图转化为图语言语料,让LLM直接学习图结构
  • 仅用少量词元即可描述目标节点的多阶结构信息
  • 适合需要高效建模复杂图结构的节点分类任务

近期工作尝试利用大语言模型(LLMs)处理带文本属性的图结构,在节点分类任务中通过描述图结构或聚合文本嵌入来实现。然而,这类方法存在两大局限:(i) 描述高阶图结构时文本冗长;(ii) 仅依赖文本属性无法充分表达图结构信息。现有方法难以在不增加负担的情况下精确建模图结构,且LLM缺乏原生图结构建模能力,难以捕捉远距离节点间的复杂依赖关系。受“预训练语言模型可在少量微调后迁移到新语言”的启发,本文提出图定义语言框架GDL4LLM,使LLM能将强大的语言理解能力迁移至图数据。该方法将图转换为图语言语料,并在此上预训练LLM,使其充分理解图结构。微调阶段,仅需少量词元即可简洁描述目标节点的结构信息。实验在三个真实数据集上验证,GDL4LLM显著优于基于描述和文本嵌入的基线方法,能高效建模不同阶次的图结构。

原文摘要 · Abstract (English)

Recent efforts leverage Large Language Models (LLMs) for modeling text-attributed graph structures in node classification tasks. These approaches describe graph structures for LLMs to understand or aggregate LLM-generated textual attribute embeddings through graph structure. However, these approaches face two main limitations in modeling graph structures with LLMs. (i) Graph descriptions become verbose in describing high-order graph structure. (ii) Textual attributes alone do not contain adequate graph structure information. It is challenging to model graph structure concisely and adequately with LLMs. LLMs lack built-in mechanisms to model graph structures directly. They also struggle with complex long-range dependencies between high-order nodes and target nodes. Inspired by the observation that LLMs pre-trained on one language can achieve exceptional performance on another with minimal additional training, we propose \textbf{G}raph-\textbf{D}efined \textbf{L}anguage for \textbf{L}arge \textbf{L}anguage \textbf{M}odel (GDL4LLM). This novel framework enables LLMs to transfer their powerful language understanding capabilities to graph-structured data. GDL4LLM translates graphs into a graph language corpus instead of graph descriptions and pre-trains LLMs on this corpus to adequately understand graph structures. During fine-tuning, this corpus describes the structural information of target nodes concisely with only a few tokens. By treating graphs as a new language, GDL4LLM enables LLMs to model graph structures adequately and concisely for node classification tasks. Extensive experiments on three real-world datasets demonstrate that GDL4LLM outperforms description-based and textual attribute embeddings-based baselines by efficiently modeling different orders of graph structure with LLMs.

图学习LLM图语言节点分类

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。