arXiv:2504.20115cs.SEcs.AI2025-04被引 24

将学术论文的图文内容自动转为可运行代码库,解决研究复现难题。

AutoP2C: An LLM-Based Agent Framework for Code Repository Generation from Multimodal Content in Academic Papers

  • 用大模型构建多智能体框架,解析论文中的文字、公式和图表。
  • 在8篇论文上全部成功生成可运行代码库,远超现有模型表现。
  • 适合科研人员快速复现论文实验,也适用于自动化研究流水线。

机器学习研究广泛发表于包含文本、图表和表格结果的学术论文中,但将这些多模态内容转化为可执行代码仍具挑战性且耗时,需大量领域知识。我们提出「论文到代码」(P2C)新任务,旨在将科学出版物的多模态内容转换为完整可运行的代码仓库,超越仅从文本生成孤立代码片段的现有范式。为实现自动化的P2C,我们提出AutoP2C——一个基于大语言模型的多智能体框架,能处理论文中的文本与视觉内容,生成完整的代码仓库。其包含四个阶段:(1) 从已有代码库提取仓库结构蓝图;(2) 融合文本、公式与图表信息进行多模态内容解析;(3) 分层任务分解以支持结构化代码生成;(4) 迭代反馈驱动的调试机制,确保功能与性能。在包含八篇论文的基准测试中,AutoP2C成功为所有论文生成可运行代码仓库,而OpenAI-o1或DeepSeek-R1仅能对其中一篇生成可运行代码。代码已开源:https://github.com/shoushouyu/Automated-Paper-to-Code。

原文摘要 · Abstract (English)

Machine Learning (ML) research is spread through academic papers featuring rich multimodal content, including text, diagrams, and tabular results. However, translating these multimodal elements into executable code remains a challenging and time-consuming process that requires substantial ML expertise. We introduce ``Paper-to-Code'' (P2C), a novel task that transforms the multimodal content of scientific publications into fully executable code repositories, which extends beyond the existing formulation of code generation that merely converts textual descriptions into isolated code snippets. To automate the P2C process, we propose AutoP2C, a multi-agent framework based on large language models that processes both textual and visual content from research papers to generate complete code repositories. Specifically, AutoP2C contains four stages: (1) repository blueprint extraction from established codebases, (2) multimodal content parsing that integrates information from text, equations, and figures, (3) hierarchical task decomposition for structured code generation, and (4) iterative feedback-driven debugging to ensure functionality and performance. Evaluation on a benchmark of eight research papers demonstrates the effectiveness of AutoP2C, which can successfully generate executable code repositories for all eight papers, while OpenAI-o1 or DeepSeek-R1 can only produce runnable code for one paper. The code is available at https://github.com/shoushouyu/Automated-Paper-to-Code.

论文生成代码复现多模态大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。