用结构化工具提升大模型在知识图谱中的推理能力。
Beyond Vector Similarity: A Structural Analysis of Graph-Augmented Retrieval for Industrial Knowledge Graphs
- 引入9种遍历算子和6种计算工具，让大模型自主选择推理路径。
- 在46节点图上，新方法在5类结构查询上准确率提升至F1=0.632。
- 揭示向量检索无法解决结构化问题，适合工业知识图谱应用者参考。
检索增强生成（RAG）在需要对互联实体进行结构推理的查询中表现不佳。我们对比了八种用于航空航天供应链情报的检索架构，从文本检索逐步推进到图遍历与图计算。基于一个包含46个节点、64条类型边的知识图谱，评估了23个跨越10类意图的查询，发现五类查询对向量检索而言在结构上不可达。核心发现是算子词汇假说：大模型图推理的障碍并非模型智能，而是可用计算算子的限制。配备9种类型遍历原语的LLM查询规划器，优于定制处理模块（F1=0.632 vs. 0.472），并能泛化到未见查询。增加6种图计算工具后，大模型可精准选择工具应对遍历失效的查询类别。此外，我们识别出测量缺口：实体级F1在综合答案正确的结构查询中系统性低估。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) fails systematically on queries requiring structural reasoning over interconnected entities. We compare eight retrieval architectures for aerospace supply chain intelligence, progressing from text retrieval through graph traversal to graph computation. Using a 46-node knowledge graph with 64 typed edges, we evaluate 23 queries across 10 intent categories and demonstrate that five query classes are structurally unreachable for vector retrieval. Our central finding is the operator vocabulary thesis: the barrier to LLM-based graph reasoning is not model intelligence but the computational operators available as tools. An LLM Query Planner with 9 typed traversal primitives outperforms bespoke handlers (F1 = 0.632 vs. 0.472) while generalizing to unseen queries. Adding 6 graph computation tools, the LLM selectively adopts them for exactly the query categories where traversal fails. We also identify a measurement gap: entity-level F1 systematically underscores structural queries where comprehensive answers are correct.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。