arXiv:2503.21223cs.LG2025-03被引 1

用树结构引导大模型,让文本图的结构学习更高效准确。

Unlocking Graph Structure Learning with Tree-Guided Large Language Models

  • 以树为媒介,让大模型在上下文中学习图结构与文本关联。
  • 在11个数据集上达到最先进性能,且无需微调、扩展性强。
  • 适合需要高效融合文本与图结构的研究者,尤其擅长多领域场景。

近期大语言模型(LLMs)的兴起推动了将文本描述融入图结构,形成文本属性图(TAGs),从数据角度增强模型编码能力。现有研究显示,图结构学习(GSL)是提升数据效用的关键技术,对高效TAG学习尤为重要。然而,多数GSL方法针对无文本的传统图设计,亟需新范式。核心挑战在于:(1) 如何在大模型参数量巨大的背景下定义合理的优化目标?(2) 如何设计能无缝集成大模型的高效架构?针对问题(1),本文将传统GSL目标重构为树优化框架,将重点从训练边预测器转为语言感知的树采样;针对问题(2),提出解耦且无需训练的模型设计原则,将重点从计算密集型微调转向高效推理。基于此,提出大型语言与树助手(LLaTA),利用基于树的大模型上下文学习,增强拓扑与文本理解,实现可靠推理并生成优化图结构。在11个数据集上的大量实验表明,LLaTA兼具灵活性(可适配任意主干网络)、可扩展性(优于其他基于大模型的GSL方法)和有效性(在多种领域数据集上达到最优预测性能)。

原文摘要 · Abstract (English)

Recently, the emergence of large language models (LLMs) has motivated integrating language descriptions into graphs, forming text-attributed graphs (TAGs) that enhance model encoding capabilities from a data-centric perspective. A review of prior advancements highlights that graph structure learning (GSL) is a pivotal technique for improving data utility, making it highly relevant to efficient TAG learning. However, most GSL methods are tailored for traditional graphs without textual information, underscoring the necessity of developing a new GSL paradigm. Despite clear motivations, it remains challenging: (1) How can we define a reasonable optimization objective for GSL in the era of LLMs, considering the massive parameters in LLMs? (2) How can we design an efficient model architecture that enables seamless integration of LLMs for this optimization objective? For Question 1, we reformulate existing GSL optimization objectives as a tree optimization framework, shifting the focus from obtaining a well-trained edge predictor to a language-aware tree sampler. For Question 2, we propose decoupled and training-free model design principles for LLM integration, shifting the focus from computation-intensive fine-tuning to more efficient inference. Based on this, we propose Large Language and Tree Assistant (LLaTA), which leverages tree-based LLM in-context learning to enhance the understanding of topology and text, enabling reliable inference and generating improved graph structure. Extensive experiments on 11 datasets demonstrate that LLaTA enjoys flexibility-incorporated with any backbone; scalability-outperforms other LLM-based GSL methods; and effectiveness-achieving SOTA predictive performance across a variety of datasets from different domains.

图神经网络大模型结构学习文本图

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。