用分阶段规划提升大模型编程准确率，代码生成更靠谱。
Planning-Driven Programming: A Large Language Model Programming Workflow
- 先写自然语言解题计划，再据此生成代码
- 在多个基准上最高提升16.4%的通过率
- 适合想提升代码生成质量的研究者和开发者
大语言模型在代码生成方面表现强劲，但现有方法依赖可见测试持续迭代，受限于模型效率与推理能力。本文提出一种结构化两阶段编程工作流（LPW），第一阶段生成并验证自然语言解题计划，第二阶段基于该计划生成初始代码并进行验证。若代码未通过测试，计划验证结果将作为修正依据，确保修复一致性。相比现有先进方法，LPW在多个主流文本到代码生成基准上显著提升性能：在HumanEval上达到98.2%的Pass@1准确率，在MBPP上达84.8%，LiveCode为59.3%，APPS为62.6%，CodeContest为34.7%，均使用GPT-4o作为主干模型。代码已开源。
原文摘要 · Abstract (English)
The strong performance of large language models (LLMs) raises extensive discussion on their application to code generation. Recent research suggests continuous program refinements through visible tests to improve code generation accuracy in LLMs. However, these methods suffer from LLMs' inefficiency and limited reasoning capacity. In this work, we propose an LLM programming workflow (LPW) designed to improve both initial code generation and subsequent refinements within a structured two-phase workflow. Specifically, the solution generation phase formulates a solution plan, which is then verified through visible tests to specify the intended natural language solution. Subsequently, the code implementation phase drafts an initial code according to the solution plan and its verification. If the generated code fails the visible tests, the plan verification serves as the intended solution to consistently inform the refinement process for correcting bugs. Compared to state-of-the-art methods across various existing LLMs, LPW significantly improves the Pass@1 accuracy by up to 16.4% on well-established text-to-code generation benchmarks. LPW also sets new state-of-the-art Pass@1 accuracy, achieving 98.2% on HumanEval, 84.8% on MBPP, 59.3% on LiveCode, 62.6% on APPS, and 34.7% on CodeContest, using GPT-4o as the backbone. Our code is publicly available at: https://github.com/you68681/lpw
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。