用可执行知识图谱提升AI研究复现成功率
What Makes AI Research Replicable? Executable Knowledge Graphs as Scientific Knowledge Representations
- 构建论文中心的可执行知识图谱，整合代码与技术细节
- 在PaperBench上提升10.9%复现准确率（o3-mini）
- 适合希望自动化复现论文的开发者与研究者
AI研究复现对大语言模型代理而言至关重要却充满挑战。现有方法常因背景知识不足及检索增强生成（RAG）无法捕捉引用论文中的隐含技术细节，难以生成可执行代码。同时，以往方法忽略实现级代码信号，缺乏支持多粒度检索与复用的结构化知识表示。为此，我们提出可插拔的论文中心知识库——可执行知识图谱（xKG），自动从文献中提取代码片段与技术洞察。在三种代理框架与两种LLM中集成xKG后，其在PaperBench上表现显著提升（使用o3-mini时达10.9%），验证了其作为通用可扩展复现解决方案的有效性。代码已开源：https://github.com/zjunlp/xKG。
原文摘要 · Abstract (English)
Replicating AI research is a crucial yet challenging task for large language model (LLM) agents. Existing approaches often struggle to generate executable code, primarily due to insufficient background knowledge and the limitations of retrieval-augmented generation (RAG) methods, which fail to capture latent technical details hidden in referenced papers. Furthermore, previous approaches tend to overlook valuable implementation-level code signals and lack structured knowledge representations that support multi-granular retrieval and reuse. To overcome these challenges, we propose Executable Knowledge Graphs (xKG), a pluggable, paper-centric knowledge base that automatically integrates code snippets and technical insights extracted from scientific literature. When integrated into three agent frameworks with two different LLMs, xKG shows substantial performance gains (10.9% with o3-mini) on PaperBench, demonstrating its effectiveness as a general and extensible solution for automated AI research replication. Code is available at https://github.com/zjunlp/xKG.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。