用相似例题引导大模型更准规划知识图谱问答路径
Exemplar-Guided Planing: Enhanced LLM Agent for KGQA
- 从训练数据中提取典型问答模板,用语义匹配找相似例题
- 在两个真实数据集上准确率提升显著,优于基线系统
- 适合想提升大模型知识图谱问答效率的研究者和开发者
大语言模型作为交互式代理在知识图谱问答中展现出巨大潜力,但常因自然语言与结构化知识图谱之间的语义鸿沟,导致规划效果不佳、探索效率低下。现有无训练方法又未能充分挖掘训练数据中的有效推理模式。为此,我们提出一种新框架——例题引导规划(EGP),通过实体模板化预处理训练集问题,统一语义变体;再利用语义嵌入与高效FAISS索引,从预处理数据中检索出高度相似的例题及其成功推理路径。这些例题动态引导大模型在两个关键阶段进行规划:(1) 任务分解时对齐生成的子目标与已验证的推理步骤;(2) 关系探索时提供高质量辅助信息,提升关系剪枝准确性。此外,在关系探索中引入智能前瞻机制,提前探索有前景路径,可能提前终止探索以提高效率。我们将EGP应用于Plan-on-Graph(PoG)框架,命名为PoG-EGP。在WebQSP和CWQ两个真实世界知识图谱问答数据集上的大量实验表明,PoG-EGP显著优于基线PoG系统及其他对比方法。
原文摘要 · Abstract (English)
Large Language Models (LLMs) as interactive agents show significant promise in Knowledge Graph Question Answering (KGQA) but often struggle with the semantic gap between natural language queries and structured knowledge graph (KG) representations. This leads to suboptimal planning and inefficient exploration on KG, while training-free approaches often underutilize valuable reasoning patterns in training data. To address these limitations, we propose a novel framework, Exemplar-Guided Planning (EGP), which enhances the planning capabilities of LLM agents for KGQA. EGP first preprocesses the training set questions via entity templating to normalize semantic variations. It then retrieves highly similar exemplary questions and their successful reasoning paths from this preprocessed set using semantic embeddings and an efficient FAISS index. These retrieved exemplars dynamically guide the LLM's planning process in two key phases: (1) Task Decomposition, by aligning generated sub-objectives with proven reasoning steps, and (2) Relation Exploration, by providing high-quality auxiliary information to improve relation pruning accuracy. Additionally, we introduce a Smart Lookahead mechanism during relation exploration to improve efficiency by preemptively exploring promising paths and potentially terminating exploration earlier. We apply EGP to the Plan-on-Graph (PoG) framework, termed PoG-EGP. Extensive experiments on two real-world KGQA datasets, WebQSP and CWQ, demonstrate that PoG-EGP significantly improves over the baseline PoG system and other compared methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。