arXiv:2412.12152cs.LGcs.AI2024-12KDD被引 22

将图推理任务拆解为三步指令,显著提升小模型图推理能力。

GraphTool-Instruction: Revolutionizing Graph Reasoning in LLMs through Decomposed Subtask Instruction

  • 把图推理拆成提取图、识别工具、提取参数三步,每步配专用指令。
  • 在80亿以下小模型上性能超越现有方法30%以上,接近GPT-4o水平。
  • 无需微调,可直接套用;适合想低成本提升图推理的开发者。

大语言模型(LLMs)已展现理解基础图属性和解决多种图推理任务的能力。现有文本指令方法性能较差。受工具学习启发,工具指令方法通过调用函数、API或模型实现显著提升,但忽视图结构信息,导致13B以下小模型表现不佳。为此,我们提出GraphTool-Instruction,将图推理任务分解为图提取、工具名识别、工具参数提取三个子任务,并为每个子任务设计专用指令。该方法无需微调,可即插即用。基于此,我们构建了包含20个图推理任务的GTools数据集,并训练出基于Llama3-8B的GraphForge模型。在20项不同图类型(如图大小、方向)的任务中,GraphTool-Instruction达到当前最优性能。在GTools上微调后,GraphForge相比增强版GPT-3.5-turbo提升超30%,性能媲美高成本的GPT-4o。代码与数据已公开。

原文摘要 · Abstract (English)

Large language models (LLMs) have been demonstrated to possess the capabilities to understand fundamental graph properties and address various graph reasoning tasks. Existing methods fine-tune LLMs to understand and execute graph reasoning tasks by specially designed task instructions. However, these Text-Instruction methods generally exhibit poor performance. Inspired by tool learning, researchers propose Tool-Instruction methods to solve various graph problems by special tool calling (e.g., function, API and model), achieving significant improvements in graph reasoning tasks. Nevertheless, current Tool-Instruction approaches focus on the tool information and ignore the graph structure information, which leads to significantly inferior performance on small-scale LLMs (less than 13B). To tackle this issue, we propose GraphTool-Instruction, an innovative Instruction-tuning approach that decomposes the graph reasoning task into three distinct subtasks (i.e., graph extraction, tool name identification and tool parameter extraction), and design specialized instructions for each subtask. Our GraphTool-Instruction can be used as a plug-and-play prompt for different LLMs without fine-tuning. Moreover, building on GraphTool-Instruction, we develop GTools, a dataset that includes twenty graph reasoning tasks, and create a graph reasoning LLM called GraphForge based on Llama3-8B. We conduct extensive experiments on twenty graph reasoning tasks with different graph types (e.g., graph size or graph direction), and we find that GraphTool-Instruction achieves SOTA compared to Text-Instruction and Tool-Instruction methods. Fine-tuned on GTools, GraphForge gets further improvement of over 30% compared to the Tool-Instruction enhanced GPT-3.5-turbo, and it performs comparably to the high-cost GPT-4o. Our codes and data are available at https://anonymous.4open.science/r/GraphTool-Instruction.

图推理指令调优小模型工具学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。