arXiv:2502.05664cs.CLcs.AI2025-02NAACL被引 52

用模拟推理提升代码生成,让AI像人一样逐步验证思路

CODESIM: Multi-Agent Code Generation and Problem Solving through Simulation-Driven Planning and Debugging

  • 通过输入输出的逐步模拟验证计划,实现自内而外的调试
  • 在7个竞赛级任务中达成新SOTA,HumanEval通过率达95.1%
  • 适合需要高精度代码生成与自动纠错的研究者和开发者

大型语言模型在代码生成与问题求解方面取得显著进展。现有方法依赖外部工具的迭代调试器,利用编译器或其他工具的运行时反馈来优化由不同方法生成的粗略程序。然而,这些方法的效果严重依赖初始代码生成质量,这一挑战仍未解决。本文提出CodeSim,一种全新的多智能体代码生成框架,通过类人的感知方式,全面覆盖程序合成-规划、编码与调试阶段。如同人类通过可视化模拟验证算法理解,CodeSim创新性地采用逐步输入输出模拟进行计划验证与内部调试。在七个具有挑战性的竞赛级问题求解与程序合成基准上进行的大量实验表明,CodeSim展现出卓越的代码生成能力。其框架在多个基准上达到新的最先进水平(pass@1):HumanEval 95.1%、MBPP 90.7%、APPS 22%、CodeContests 29.1%。此外,该方法在与外部调试器级联后展现出更大的提升潜力。为促进该领域的进一步研究与发展,我们已开源该框架(https://kagnlp.github.io/codesim.github.io/)。

原文摘要 · Abstract (English)

Large Language Models (LLMs) have made significant strides in code generation and problem solving. Current approaches employ external tool-based iterative debuggers that use compiler or other tool-based runtime feedback to refine coarse programs generated by various methods. However, the effectiveness of these approaches heavily relies on the quality of the initial code generation, which remains an open challenge. In this paper, we introduce CodeSim, a novel multi-agent code generation framework that comprehensively addresses the stages of program synthesis-planning, coding, and debugging-through a human-like perception approach. As human verifies their understanding of any algorithms through visual simulation, CodeSim uniquely features a method of plan verification and internal debugging through the step-by-step simulation of input/output. Extensive experiments across seven challenging competitive problem-solving and program synthesis benchmarks demonstrate CodeSim's remarkable code generation capabilities. Our framework achieves new state-of-the-art (pass@1) results-(HumanEval 95.1%, MBPP 90.7%, APPS 22%, and CodeContests 29.1%). Furthermore, our method shows potential for even greater enhancement when cascaded with external debuggers. To facilitate further research and development in this area, we have open-sourced our framework in this link (https://kagnlp.github.io/codesim.github.io/).

代码生成多智能体模拟推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。