arXiv:2410.19084cs.CL2024-10被引 15

GCoder让大模型更懂图计算,准确率比GPT-4o高16.42%。

GCoder: Improving Large Language Model for Generalized Graph Problem Solving

  • 用代码生成代替推理步骤,提升可验证性与泛化能力。
  • 在多类图问题上平均准确率提升16.42%,支持百万级节点图。
  • 适合需要处理复杂图结构的开发者和研究者使用。

大语言模型(LLM)展现出强大推理能力,适用于图计算等复杂任务。传统基于推理步骤的方法存在不可验证、长时推理能力弱及对图结构变化泛化差等问题。为此,我们提出GCoder——一种基于代码的LLM,用于增强通用图计算问题求解能力。方法包括构建包含多样图格式与算法的大型训练数据集GraphWild,采用监督微调(SFT)与编译器反馈强化学习(RLCF)的多阶段训练流程,并在未见任务中引入混合检索技术提升性能。实验表明,GCoder在各类图计算问题上平均准确率较GPT-4o提升16.42%。同时,该模型能高效处理含数百万节点的大规模图及多种输入格式,突破了以往以推理步骤为核心的模型局限。此进展为更直观高效的图问题求解提供了新路径。代码与数据见:https://github.com/Bklight999/WWW25-GCoder/tree/master。

原文摘要 · Abstract (English)

Large Language Models (LLMs) have demonstrated strong reasoning abilities, making them suitable for complex tasks such as graph computation. Traditional reasoning steps paradigm for graph problems is hindered by unverifiable steps, limited long-term reasoning, and poor generalization to graph variations. To overcome these limitations, we introduce GCoder, a code-based LLM designed to enhance problem-solving in generalized graph computation problems. Our method involves constructing an extensive training dataset, GraphWild, featuring diverse graph formats and algorithms. We employ a multi-stage training process, including Supervised Fine-Tuning (SFT) and Reinforcement Learning from Compiler Feedback (RLCF), to refine model capabilities. For unseen tasks, a hybrid retrieval technique is used to augment performance. Experiments demonstrate that GCoder outperforms GPT-4o, with an average accuracy improvement of 16.42% across various graph computational problems. Furthermore, GCoder efficiently manages large-scale graphs with millions of nodes and diverse input formats, overcoming the limitations of previous models focused on the reasoning steps paradigm. This advancement paves the way for more intuitive and effective graph problem-solving using LLMs. Code and data are available at here: https://github.com/Bklight999/WWW25-GCoder/tree/master.

图计算大模型代码生成强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。