研究者代理自我优化,提升知识图谱问答准确率
Towards Researcher Agents for Knowledge-Graph Question Answering

- 自研代理每轮推理后自动调整提示词与代码,实现闭环改进
- 在DBpedia上达0.22整体准确率,且快速收敛
- 揭示谓词选择是主要瓶颈,建议新基准融合多维度评估
将自然语言问题转化为可在大型知识图谱上执行的SPARQL查询,需解决词汇歧义、表面术语与目标本体的关联,以及生成语法正确且语义忠实的图模式。本文提出一种超越静态工具调用代理的研究者代理:该代理在每次验证集推理后,自主提出并测试对其提示词、规则及工具编排代码的修改。我们在DBpedia上实现了九个版本的迭代演化,使用低成本推理模型进行驱动,并部署表现最佳配置于两个更强的主干模型。研究发现:(i) 自我改进快速收敛,在2025年DBpedia验证集上达到0.22的整体准确率;(ii) 瓶颈始终在于基础图模式的谓词选择,而非SPARQL语法或修饰符;(iii) 部分基准题目因DBpedia中的属性歧义惩罚正确查询,提示未来文本到SPARQL基准应结合机器翻译与信息检索指标评分。
原文摘要 · Abstract (English)
Translating a natural-language question into a SPARQL query that can be executed against a large knowledge graph requires resolving lexical ambiguity, grounding surface terms in the target ontology, and producing graph patterns that are both syntactically valid and semantically faithful. We present an agentic text-to-SPARQL system that goes one step beyond static tool-using agents: a researcher agent that, after each round of inference on a validation set, proposes and tests changes to its own prompts, rules, and tool-orchestration code. We instantiate the loop on DBpedia, evolve nine successive versions of the agent driven by a low-cost reasoning model, and deploy the best-performing configuration with two stronger backbone models. The study yields three observations: (i) self-improvement converges quickly and then achieves 0.22 overall accuracy on the 2025 DBpedia validation set; (ii) the bottleneck is consistently in basic-graph-pattern predicate selection, not in SPARQL syntax or modifiers; and (iii) several benchmark items appear to penalise correct queries due to property ambiguity in DBpedia, suggesting that future Text-to-SPARQL benchmarks should be scored using a combination of machine translation and information retrieval metrics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。