用分层多智能体系统自动复现论文代码并执行，效果更稳。
HiRAS: A Hierarchical Multi-Agent Framework for Paper-to-Code Generation and Execution
- 分层管理器协调多个专业智能体，按细粒度阶段协作
- 相比之前方法，代码复现准确率提升超10%，幻觉显著减少
- 新增评估协议，更贴近真实实验复现场景，适合研究自动化者
大型语言模型在自动化计算研究方面展现出巨大潜力，尤其是在复现实验结果方面。然而，现有方法普遍采用固定顺序的智能体流水线，全局协调能力弱，限制了其鲁棒性和整体性能。本文提出分层研究智能体系统（HiRAS），一个用于端到端实验复现的分层多智能体框架，通过监督管理智能体协调各细粒度阶段的专业智能体。我们还发现当前Paper2Code基准中无参考评估存在缺陷，因此引入改进协议Paper2Code-Extra（P2C-Ex），融合仓库级信息，更好地对齐原始基于参考的评估指标。通过大量实验验证，所提方法在使用开源骨干模型时，相对之前最优方法提升超过10%的性能，并显著降低评估中的幻觉现象。项目已开源：https://github.com/KOU-199024/HiRAS。
原文摘要 · Abstract (English)
Recent advances in large language models have highlighted their potential to automate computational research, particularly reproducing experimental results. However, existing approaches still use fixed sequential agent pipelines with weak global coordination, which limits their robustness and overall performance. In this work, we propose Hierarchical Research Agent System (HiRAS), a hierarchical multi-agent framework for end-to-end experiment reproduction that employs supervisory manager agents to coordinate specialised agents across fine-grained stages. We also identify limitations in the reference-free evaluation of the Paper2Code benchmark and introduce Paper2Code-Extra (P2C-Ex), a refined protocol that incorporates repository-level information and better aligns with the original reference-based metric. We conduct extensive evaluation, validating the effectiveness and robustness of our proposed methods, and observing improvements, including >10\% relative performance gain beyond the previous state-of-the-art using open-source backbone models and significantly reduced hallucination in evaluation. Our work is available on GitHub: https://github.com/KOU-199024/HiRAS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。