让大模型更好理解数据库关系,提升复杂推理能力。
Large Language Models are Good Relational Learners
- 用图神经网络提取实体周边关系,生成结构化提示。
- 在多个关系学习任务上超越现有方法,提升推理准确率。
- 适合需要处理结构化数据的AI研究者和开发者。
大型语言模型(LLMs)在多个领域表现出色,但其在关系深度学习(RDL)中的应用仍不充分。现有方法通过遍历数据库中实体间的关系链接,将结构化数据转为扁平文本,但此类文本序列化会忽略关键关系结构,引入冗余,并常超出标准LLM上下文长度。本文提出Rel-LLM,一种新架构,在检索增强生成(RAG)框架下,利用基于图神经网络(GNN)的编码器,为LLMs生成结构化的关系提示。GNN编码器提取目标实体周围的局部子图,生成包含相关关系与时间依赖性的特征表示。这些表示通过去规范化过程转化为结构化提示,使LLM能有效处理复杂实体关系。大量实验表明,Rel-LLM在关键RDL任务上优于现有方法,提供了一种可扩展、高效的LLM与结构化数据融合方案。代码已公开于https://github.com/smiles724/Rel-LLM。
原文摘要 · Abstract (English)
Large language models (LLMs) have demonstrated remarkable capabilities across various domains, yet their application to relational deep learning (RDL) remains underexplored. Existing approaches adapt LLMs by traversing relational links between entities in a database and converting the structured data into flat text documents. Still, this text-based serialization disregards critical relational structures, introduces redundancy, and often exceeds standard LLM context lengths. We introduce Rel-LLM, a novel architecture that utilizes a graph neural network (GNN)- based encoder to generate structured relational prompts for LLMs within a retrieval-augmented generation (RAG) framework. Unlike traditional text-based serialization approaches, our method preserves the inherent relational structure of databases while enabling LLMs to effectively process and reason over complex entity relationships. Specifically, the GNN encoder extracts a local subgraph around an entity to build feature representations that contain relevant entity relationships and temporal dependencies. These representations are transformed into structured prompts using a denormalization process, effectively allowing the LLM to reason over relational structures. Through extensive experiments, we demonstrate that Rel-LLM outperforms existing methods on key RDL tasks, offering a scalable and efficient approach to integrating LLMs with structured data sources. Code is available at https://github.com/smiles724/Rel-LLM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。