用智能体动态控制代码生成，让大模型自己检查正确性
QualityFlow: An Agentic Workflow for Program Synthesis Controlled by LLM Quality Checks
- 设计多个智能体模拟开发流程，分阶段生成、测试和调试代码
- 在四个基准上达到当前最优，正确率显著高于已有方法
- 适合需要高可靠性代码生成的开发者或自动化测试场景
我们提出 QualityFlow，一种用于程序合成的动态智能体工作流。给定编程问题的自然语言描述和一组单元测试，目标是生成能解决问题并通过测试的正确程序。QualityFlow 包含类似软件开发团队的大语言模型（LLM）智能体，涵盖代码生成、测试与自调试。我们提出 LLM 质量检查器，能显式‘想象’生成程序的执行是否符合单元测试。质量检查动态调控工作流，包括提交最终答案、澄清问题陈述或回退先前步骤。实验表明，该检查器可精准接受正确程序，缓解错误测试生成，并防止潜在流程偏离。QualityFlow 在四个程序合成基准上取得当前最优结果：MBPP、HumanEval，以及更严格的 MBPP-EvalPlus 与 HumanEval-EvalPlus。
原文摘要 · Abstract (English)
We introduce QualityFlow, a dynamic agentic workflow for program synthesis. Given the English description of a programming problem and a set of unit tests, the model's goal is to synthesize the correct program that solves the problem and passes the tests. QualityFlow includes large language model (LLM) agents resembling a software development team, including code generation, testing, and self-debugging. We propose the LLM Quality Checker, which explicitly "imagines" whether the synthesized programs' execution would conform to the unit tests. The Quality Checks dynamically control the workflow, including actions to submit the final answer, clarify the problem statement, and revert previous workflow steps. Our experiments show that the Quality Checker can precisely accept any correct program, mitigate faulty synthesized tests, and prevent potential workflow deviation. QualityFlow establishes the state-of-the-art results on four program synthesis benchmarks: MBPP, HumanEval, and stricter evaluations from MBPP-EvalPlus and HumanEval-EvalPlus.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。