arXiv:2504.17192cs.CL2025-04被引 80

将机器学习论文自动转为可运行代码,提升研究复现效率。

Paper2Code: Automating Code Generation from Scientific Papers in Machine Learning

  • 分三阶段多智能体协作,规划、分析、生成代码全流程自动化。
  • 在PaperBench上超越主流基线,作者验证代码忠实度高。
  • 适合需快速复现论文或构建新系统的研究人员使用。

尽管机器学习研究迅速发展,但相应代码实现常缺失,导致研究人员复现结果和继承工作耗时费力。近期大型语言模型(LLMs)在理解科学文档和生成高质量代码方面表现优异。受此启发,我们提出PaperCoder,一个用于将机器学习论文转化为可运行代码仓库的多智能体框架。PaperCoder分为三个阶段:规划阶段生成高层路线图、设计系统架构并绘制图表、识别文件依赖关系、生成配置文件;分析阶段聚焦于解析具体实现细节;生成阶段则产出模块化、依赖感知的代码。每个阶段由专门设计的智能体协同完成。我们在基于模型与人工评估(包括论文作者自身评价)的实验中评估PaperCoder,以作者发布的代码库为真实标准。结果表明,PaperCoder能生成高质量且忠实的实现。此外,其在最新发布的PaperBench基准测试中持续领先,显著优于现有强基线。代码已开源:https://github.com/going-doer/Paper2Code。

原文摘要 · Abstract (English)

Despite the rapid growth of machine learning research, corresponding code implementations are often unavailable, making it slow and labor-intensive for researchers to reproduce results and build upon prior work. In the meantime, recent Large Language Models (LLMs) excel at understanding scientific documents and generating high-quality code. Inspired by this, we introduce PaperCoder, a multi-agent LLM framework that transforms machine learning papers into operational code repositories. PaperCoder operates in three stages: planning, where it constructs a high-level roadmap, designs the system architecture with diagrams, identifies file dependencies, and generates configuration files; analysis, which focuses on interpreting implementation-specific details; and generation, where modular, dependency-aware code is produced. Moreover, each phase is instantiated through a set of specialized agents designed to collaborate effectively across the pipeline. We then evaluate PaperCoder on generating code implementations from machine learning papers based on both model-based and human evaluations, particularly from the authors of those papers, with author-released repositories as ground truth if available. Our results demonstrate the effectiveness of PaperCoder in creating high-quality, faithful implementations. Furthermore, it consistently shows strengths in the recently released PaperBench benchmark, surpassing strong baselines by substantial margins. Code is available at: https://github.com/going-doer/Paper2Code.

代码生成多智能体论文复现

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。