用规划机制提升LLM查知识图谱的准确性和可靠性。
PersonalAI 2.0: Enhancing knowledge graph traversal/retrieval with planning mechanism for Personalized LLM Agents
- 引入动态多阶段查询流程，根据实体和线索迭代搜索。
- 相比基线方法，事实正确率平均提升4%，幻觉减少18%。
- 适合需要精准推理与个性化服务的AI应用开发。
我们提出PersonalAI 2.0（PAI-2），一种通过集成外部知识图谱（KG）增强大语言模型（LLM）系统的新框架。针对现有图检索增强生成（GraphRAG）方法的局限性，PAI-2引入动态、多阶段查询处理管道，实现基于提取实体、匹配图顶点和生成线索查询的自适应迭代信息搜索。在六个基准测试（Natural Questions、TriviaQA、HotpotQA、2WikiMultihopQA、MuSiQue 和 DiaASQ）上的评估显示，相较于LightRAG、RAPTOR和HippoRAG 2等方法，PAI-2在事实正确性上取得提升。在四个基准上，通过LLM-as-a-Judge评估，平均得分提高4%，表明其有效降低幻觉率并提升精度。使用图遍历算法（如BeamSearch、WaterCircles）相比标准扁平化检索器平均提升6%；启用搜索规划机制后，在六个数据集上相较禁用版本提升18%。消融实验进一步表明，PAI-2在MINE-1基准上达到89%的信息保留率，使用7-14B规模的LLM。这些结果凸显了PAI-2作为下一代个性化AI应用基础模型的潜力，具备可扩展、上下文感知的知识表征与推理能力。
原文摘要 · Abstract (English)
We introduce PersonalAI 2.0 (PAI-2), a novel framework, designed to enhance large language model (LLM) based systems through integration of external knowledge graphs (KG). The proposed approach addresses key limitations of existing Graph Retrieval-Augmented Generation (GraphRAG) methods by incorporating a dynamic, multistage query processing pipeline. The central point of PAI-2 design is its ability to perform adaptive, iterative information search, guided by extracted entities, matched graph vertices and generated clue-queries. Conducted evaluation over six benchmarks (Natural Questions, TriviaQA, HotpotQA, 2WikiMultihopQA, MuSiQue and DiaASQ) demonstrates improvement in factual correctness of generating answers compared to analogues methods (LightRAG, RAPTOR, and HippoRAG 2). PAI-2 achieves 4% average gain by LLM-as-a-Judge across four benchmarks, reflecting its effectiveness in reducing hallucination rates and increasing precision. We show that use of graph traversal algorithms (e.g. BeamSearch, WaterCircles) gain superior results compared to standard flatten retriever on average 6%, while enabled search plan enhancement mechanism gain 18% boost compared to disabled one by LLM-as-a-Judge across six datasets. In addition, ablation study reveals that PAI-2 achieves the SOTA result on MINE-1 benchmark, achieving 89% information-retention score, using LLMs from 7-14B tiers. Collectively, these findings underscore the potential of PAI-2 to serve as a foundational model for next-generation personalized AI applications, requiring scalable, context-aware knowledge representation and reasoning capabilities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。