arXiv:2501.15755cs.LG2025-01NAACL被引 24

用精心设计的提示词让通用大模型搞定图学习,效果超过专门模型。

GraphICL: Unlocking Graph Learning Potential in LLMs through Structured Prompt Design

  • 设计新提示模板,让大模型理解图结构和有限标签。
  • 在资源受限和跨领域任务中,通用模型超越专用图模型。
  • 无需训练,靠提示工程就能激发大模型图学习潜力,适合研究者参考。

随着文本与关系系统的重要性提升,增强大语言模型(LLMs)处理图结构数据的能力成为研究热点,尤其是文本属性图(TAGs),其中样本由文本描述通过边连接。尽管现有研究多聚焦于通过任务特定指令微调开发专用图LLM,但缺乏仅通过提示设计评估LLMs的全面基准。由于缺少精心设计的评估基准,大多数甚至全部定制化图LLM都与通用LLM使用简单查询(如零样本推理的LLaMA)进行比较,可能掩盖其优势与潜在问题。为实现更通用的评估并揭示LLM在图任务中的真实潜力,我们引入图上下文学习(GraphICL)基准,包含新型提示模板,能捕捉图结构并处理标签信息有限的情况。系统评估显示,配备GraphICL的通用大模型在资源受限设置和跨域任务中优于最先进的专用图LLM和图神经网络模型。这些发现表明,提示工程可在不训练的前提下显著提升LLM在图学习任务中的表现,并为图LLM研究提供强有力基线。

原文摘要 · Abstract (English)

The growing importance of textual and relational systems has driven interest in enhancing large language models (LLMs) for graph-structured data, particularly Text-Attributed Graphs (TAGs), where samples are represented by textual descriptions interconnected by edges. While research has largely focused on developing specialized graph LLMs through task-specific instruction tuning, a comprehensive benchmark for evaluating LLMs solely through prompt design remains surprisingly absent. Without such a carefully crafted evaluation benchmark, most if not all, tailored graph LLMs are compared against general LLMs using simplistic queries (e.g., zero-shot reasoning with LLaMA), which can potentially camouflage many advantages as well as unexpected predicaments of them. To achieve more general evaluations and unveil the true potential of LLMs for graph tasks, we introduce Graph In-context Learning (GraphICL) Benchmark, a comprehensive benchmark comprising novel prompt templates designed to capture graph structure and handle limited label knowledge. Our systematic evaluation shows that general-purpose LLMs equipped with our GraphICL outperform state-of-the-art specialized graph LLMs and graph neural network models in resource-constrained settings and out-of-domain tasks. These findings highlight the significant potential of prompt engineering to enhance LLM performance on graph learning tasks without training and offer a strong baseline for advancing research in graph LLMs.

图学习提示工程大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。