arXiv:2506.10014cs.LG2025-06

用自然语言描述节点,让大模型无须消息传递就能处理图数据。

NOCL: Node-Oriented Conceptualization LLM for Graph Tasks without Message Passing

  • 将节点属性转为结构化文本,扩展大模型适用范围
  • 节点概念编码使令牌长度减少93.9%
  • 支持零样本泛化,适合少标签场景

图在社交网络、生物和推荐系统等领域的复杂交互建模中至关重要。传统图神经网络(如MPNN)依赖监督学习,在标签稀缺时泛化能力受限;近期自监督方法仍需标注微调,难以实现零样本应用。大语言模型虽擅长自然语言任务,但在图数据上面临推理能力保持、长序列处理及仅限文本属性图的挑战。为此,我们提出节点导向的概念化大模型(NOCL),采用两项核心技术:1)节点描述,将异构节点属性转化为结构化自然语言,拓展大模型从文本属性图(TAGs)到非文本属性图的应用;2)节点概念,利用预训练语言模型将节点描述压缩为紧凑语义嵌入,相比直接使用描述,令牌长度减少高达93.9%。此外,通过图表示描述符,统一多层级图任务为共享语言查询格式,开辟图基础模型新路径。实验表明,NOCL在监督任务中性能媲美传统MPNN与混合式LLM-MPNN方法,并在零样本设置下展现更优泛化能力。

原文摘要 · Abstract (English)

Graphs are essential for modeling complex interactions across domains such as social networks, biology, and recommendation systems. Traditional Graph Neural Networks, particularly Message Passing Neural Networks (MPNNs), rely heavily on supervised learning, limiting their generalization and applicability in label-scarce scenarios. Recent self-supervised approaches still require labeled fine-tuning, limiting their effectiveness in zero-shot scenarios. Meanwhile, Large Language Models (LLMs) excel in natural language tasks but face significant challenges when applied to graphs, including preserving reasoning abilities, managing extensive token lengths from rich node attributes, and being limited to textual-attributed graphs (TAGs) and a single level task. To overcome these limitations, we propose the Node-Oriented Conceptualization LLM (NOCL), a novel framework that leverages two core techniques: 1) node description, which converts heterogeneous node attributes into structured natural language, extending LLM from TAGs to non-TAGs; 2) node concept, which encodes node descriptions into compact semantic embeddings using pretrained language models, significantly reducing token lengths by up to 93.9% compared to directly using node descriptions. Additionally, our NOCL employs graph representation descriptors to unify graph tasks at various levels into a shared, language-based query format, paving a new direction for Graph Foundation Models. Experimental results validate NOCL's competitive supervised performance relative to traditional MPNNs and hybrid LLM-MPNN methods and demonstrate superior generalization in zero-shot settings.

图神经网络大模型零样本自然语言

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。