用执行反馈同时训练代码与测试生成，提升代码质量与测试覆盖率。
GenX: Mastering Code and Test Generation with Execution Feedback
- 双模型协同训练，利用执行反馈优化代码与测试生成
- 在APPS数据集上实现更高准确率的正确代码与测试合成
- 适合需要自动测试生成的开发场景，尤其适用于缺乏测试用例的项目
近期语言建模进展使得自然语言可转化为代码，并通过执行反馈改进生成效果。然而，现有方法常依赖已有测试用例，而这些用例可能不可用或不完整。本文提出一种新方法，同时训练代码生成模型与测试生成模型，利用执行反馈优化两者性能。引入两种测试与代码数据增强策略，以及新的代码与测试评分函数。在APPS数据集上实验表明，该方法能有效生成并扩充测试用例，筛选并合成正确代码解，并对生成结果进行质量排序。当迭代训练中逐步增加测试用例与代码解数量时，模型性能优于仅在原始数据集上训练的模型。
原文摘要 · Abstract (English)
Recent advancements in language modeling have enabled the translation of natural language into code, and the use of execution feedback to improve code generation. However, these methods often rely heavily on pre-existing test cases, which may not always be available or comprehensive. In this work, we propose a novel approach that concurrently trains a code generation model and a test generation model, utilizing execution feedback to refine and enhance the performance of both. We introduce two strategies for test and code data augmentation and a new scoring function for code and test ranking. We experiment on the APPS dataset and demonstrate that our approach can effectively generate and augment test cases, filter and synthesize correct code solutions, and rank the quality of generated code and tests. The results demonstrate that our models, when iteratively trained with an increasing number of test cases and code solutions, outperform those trained on the original dataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。