用多模型协作自动调试代码,让大模型像程序员一样迭代优化。
RGD: Multi-LLM Based Agent Debugger via Refinement and Generation Guidance
- 设计三类模型分工:引导、调试、反馈,分步生成并自修正代码。
- 在HumanEval上比顶尖方法高9.8%,MBPP上高16.2%。
- 适合想提升代码生成质量的开发者和研究者。
大型语言模型(LLMs)在代码生成任务中展现出巨大潜力,提示工程也提升了其对文本信息的理解能力。然而,确保生成代码的准确性通常仍需程序员进行大量测试与验证。尽管LLMs能根据任务描述生成代码,但其准确性在复杂任务中受限,主要因需同时理解文本并生成语法语义正确的代码,且缺乏自动优化能力。真实开发中,程序员常通过反复反馈与调试来完善程序。受此启发,我们提出一种基于多LLM的代码生成与自动调试框架——精炼与引导调试(RGD)。RGD采用三个独立的LLM代理:引导代理、调试代理与反馈代理,将代码生成拆解为多个步骤,实现基于自我反思与反馈的迭代优化。实验表明,RGD在代码生成方面表现卓越,在HumanEval数据集上较现有最佳方法提升9.8%,在MBPP数据集上提升16.2%,显著优于主流方法与传统直接提示方式。结果证明该框架有效增强了LLMs自主生成与优化代码的能力。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have shown incredible potential in code generation tasks, and recent research in prompt engineering have enhanced LLMs' understanding of textual information. However, ensuring the accuracy of generated code often requires extensive testing and validation by programmers. While LLMs can typically generate code based on task descriptions, their accuracy remains limited, especially for complex tasks that require a deeper understanding of both the problem statement and the code generation process. This limitation is primarily due to the LLMs' need to simultaneously comprehend text and generate syntactically and semantically correct code, without having the capability to automatically refine the code. In real-world software development, programmers rarely produce flawless code in a single attempt based on the task description alone, they rely on iterative feedback and debugging to refine their programs. Inspired by this process, we introduce a novel architecture of LLM-based agents for code generation and automatic debugging: Refinement and Guidance Debugging (RGD). The RGD framework is a multi-LLM-based agent debugger that leverages three distinct LLM agents-Guide Agent, Debug Agent, and Feedback Agent. RGD decomposes the code generation task into multiple steps, ensuring a clearer workflow and enabling iterative code refinement based on self-reflection and feedback. Experimental results demonstrate that RGD exhibits remarkable code generation capabilities, achieving state-of-the-art performance with a 9.8% improvement on the HumanEval dataset and a 16.2% improvement on the MBPP dataset compared to the state-of-the-art approaches and traditional direct prompting approaches. We highlight the effectiveness of the RGD framework in enhancing LLMs' ability to generate and refine code autonomously.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。