让大模型真正读懂图数据，提升图分析性能。
LoReC: Rethinking Large Language Models for Graph Data Analysis
- 通过看、记、对比三阶段增强大模型对图结构的理解。
- 在多个数据集上超越现有图大模型方法，部分优于传统GNN。
- 插件式设计，可无缝接入现有大模型，适合图分析研究者。
大语言模型（LLM）的兴起重塑了图数据交互方式，催生了图大模型（GraphLLM）新范式。尽管已有研究显示图学习可受益于LLM，但直接使用LLM进行图任务预测时效果有限，甚至不如传统GNN方法。我们深入分析发现，问题源于LLM处理图数据能力不足及忽略图信息的倾向。为此，提出LoReC（Look, Remember, and Contrast）——一种新型即插即用方法，分三步增强LLM对图数据的理解：(1) 看：重新分配注意力聚焦图结构；(2) 记：将图信息重注入前馈网络（FFN）；(3) 对比：修正解码过程中的原始输出。大量实验表明，LoReC显著优于当前GraphLLM方法，并在多种数据集上超越基于GNN的方法。代码已开源：https://github.com/Git-King-Zhan/LoReC。
原文摘要 · Abstract (English)
The advent of Large Language Models (LLMs) has fundamentally reshaped the way we interact with graphs, giving rise to a new paradigm called GraphLLM. As revealed in recent studies, graph learning can benefit from LLMs. However, we observe limited benefits when we directly utilize LLMs to make predictions for graph-related tasks within GraphLLM paradigm, which even yields suboptimal results compared to conventional GNN-based approaches. Through in-depth analysis, we find this failure can be attributed to LLMs' limited capability for processing graph data and their tendency to overlook graph information. To address this issue, we propose LoReC (Look, Remember, and Contrast), a novel plug-and-play method for GraphLLM paradigm, which enhances LLM's understanding of graph data through three stages: (1) Look: redistributing attention to graph; (2) Remember: re-injecting graph information into the Feed-Forward Network (FFN); (3) Contrast: rectifying the vanilla logits produced in the decoding process. Extensive experiments demonstrate that LoReC brings notable improvements over current GraphLLM methods and outperforms GNN-based approaches across diverse datasets. The implementation is available at https://github.com/Git-King-Zhan/LoReC.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。