自动迭代修复大模型生成代码与测试，提升质量与可靠性。
LLMLOOP: Improving LLM-Generated Code and Tests through Automated Iterative Feedback Loops
- 通过五轮迭代反馈自动修复编译错误与静态分析问题
- 在HUMANEVAL-X上显著降低测试失败率，生成高质量测试用例
- 适合开发人员快速验证和优化AI生成的代码与测试
大型语言模型（LLMs）在生成源代码方面表现卓越，但生成的代码常存在编译错误或逻辑错误。研究人员和开发者需反复手动检查和修正，浪费大量精力。本文提出LLMLOOP框架，自动化地对LLM生成的代码和测试用例进行迭代优化。该框架包含五个循环：解决编译错误、处理静态分析问题、修复测试用例失败、通过变异分析提升测试质量。这些循环确保生成的测试用例具备高覆盖率并可作为代码验证与回归测试机制。我们在HUMANEVAL-X这一最新编程任务基准上评估了LLMLOOP，结果表明其能有效提升生成代码与测试的质量。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are showing remarkable performance in generating source code, yet the generated code often has issues like compilation errors or incorrect code. Researchers and developers often face wasted effort in implementing checks and refining LLM-generated code, frequently duplicating their efforts. This paper presents LLMLOOP, a framework that automates the refinement of both source code and test cases produced by LLMs. LLMLOOP employs five iterative loops: resolving compilation errors, addressing static analysis issues, fixing test case failures, and improving test quality through mutation analysis. These loops ensure the generation of high-quality test cases that serve as both a validation mechanism and a regression test suite for the generated code. We evaluated LLMLOOP on HUMANEVAL-X, a recent benchmark of programming tasks. Results demonstrate the tool's effectiveness in refining LLM-generated outputs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。