用LLM动态导航知识图谱，证据分散时效果更优
RLM-on-KG: Heuristics First, LLMs When Needed: Adaptive Retrieval Control over Mention Graphs for Scattered Evidence
- 将LLM作为自主导航器，在查询时进行多跳实体探索
- 证据分散时LLM提升达+3.21pp，集中时仅+1.85pp
- 适合需要深度探索的复杂问答，尤其证据分布零散场景
我们通过RLM-on-KG研究：在何种情况下LLM控制器优于基于规则的遍历？该系统将LLM视为在RDF编码提及图上自主导航的智能体，用于精准问答。不同于依赖离线索引的GraphRAG，RLM-on-KG在查询时进行实体优先、多跳探索，采用确定性图构建和固定工具集。核心发现是条件性优势：LLM控制的价值取决于证据分散程度与工具调用复杂度。在GraphRAG-Bench Novel（519个问题）上，Gemini 2.0 Flash相较规则基线提升+2.47pp F1（p<0.0001），但仅比GraphRAG-local提升+0.16pp（不显著）。使用更强控制器Claude Haiku 4.5时，相比基线提升+4.37pp（p<0.001），并显著超越GraphRAG-local（+2.42pp，p<0.001）。证据分布在6-10个块时增益最大（+3.21pp），集中时最小（+1.85pp）。MuSiQue跨尺度验证表明该优势可迁移，但在小规模图上衰减明显。核心架构洞察是候选发现与排序分离：LLM贡献于探索广度，最终证据选择由纯向量重排处理。此外，探索轨迹可作为结构化数据质量的应力测试工具，诊断覆盖率、连通性、来源和可查询性。
原文摘要 · Abstract (English)
When does an LLM controller outperform rule-based traversal for knowledge graph exploration? We study this question through RLM-on-KG, a retrieval system that treats an LLM as an autonomous navigator over an RDF-encoded mention graph for grounded question answering. Unlike GraphRAG pipelines that rely on offline LLM indexing, RLM-on-KG performs entity-first, multi-hop exploration at query time using deterministic graph construction and a fixed tool set. Our central finding is a conditional advantage: the value of LLM control depends on evidence scatter and tool-calling sophistication. The paper's core claim is LLM control versus heuristic traversal, not a generic win over GraphRAG. On GraphRAG-Bench Novel (519 questions), Gemini 2.0 Flash achieves +2.47 pp F1 over a rule-based heuristic baseline (p < 0.0001), but only +0.16 pp over a GraphRAG-local variant (not significant). With a stronger controller, Claude Haiku 4.5, the gain over heuristic grows to +4.37 pp (p < 0.001) and extends to a +2.42 pp significant improvement over GraphRAG-local (p < 0.001). The gain is largest when gold evidence is scattered across 6-10 chunks (+3.21 pp) and smallest for concentrated evidence (+1.85 pp). Cross-scale validation on MuSiQue confirms that the LLM-over-heuristic advantage transfers, with expected attenuation on smaller per-question graphs. The core architectural insight is the separation of candidate discovery from ranking: the LLM adds value through exploration breadth, while final evidence selection is best handled by pure vector re-ranking. Beyond retrieval, exploration traces provide a proposed stress-test harness for structured data quality, yielding diagnostics for coverage, connectivity, provenance, and queryability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。