自动复现论文实验，用文献关系链挖掘知识
AutoReproduce: Automatic AI Experiment Reproduction with Paper Lineage
- 基于引用文献构建论文谱系，驱动自动化复现
- 在两个基准上复现准确率超基线，执行成功率显著提升
- 适合需要快速验证论文的科研人员和开发者
高效复现研究论文对加速科学进步至关重要。然而，方法复杂度不断提升，导致复现工作耗时且依赖深厚领域知识。为此，我们提出论文谱系，系统挖掘引用文献中的隐含知识。该算法构成我们提出的\ours——一个用于端到端自动复现实验代码的多智能体框架。为确保代码可执行，\ours采用基于采样的单元测试策略实现快速验证。为评估复现能力，我们引入\ourbench基准，包含经验证的实现方案及全面的评估指标。在PaperBench和\ourbench上的大量实验表明，\ours在所有指标上均持续优于现有基线，尤其在复现保真度和最终执行性能上取得显著提升。代码已公开于https://github.com/AI9Stars/AutoReproduce。
原文摘要 · Abstract (English)
Efficient reproduction of research papers is pivotal to accelerating scientific progress. However, the increasing complexity of proposed methods often renders reproduction a labor-intensive endeavor, necessitating profound domain expertise. To address this, we introduce the paper lineage, which systematically mines implicit knowledge from the cited literature. This algorithm serves as the backbone of our proposed \ours, a multi-agent framework designed to autonomously reproduce experimental code in a complete, end-to-end manner. To ensure code executability, \ours incorporates a sampling-based unit testing strategy for rapid validation. To assess reproduction capabilities, we introduce \ourbench, a benchmark featuring verified implementations, alongside comprehensive metrics for evaluating both reproduction and execution fidelity. Extensive evaluations on PaperBench and \ourbench demonstrate that \ours consistently surpasses existing baselines across all metrics. Notably, it yields substantial improvements in reproduction fidelity and final execution performance. The code is available at https://github.com/AI9Stars/AutoReproduce.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。