用强化学习优化LLM生成的单元测试质量，减少代码异味。
Reinforcement Learning from Automatic Feedback for High-Quality Unit Test Generation
- 基于静态分析指标设计奖励模型，引导LLM生成更高质量测试。
- 测试质量提升最高达23%，语法正确率接近100%。
- 适合希望提升自动化测试可靠性的开发团队使用。
软件测试是开发中关键但耗时的环节，近年来大语言模型（LLMs）在自动化测试用例生成中受到关注。然而，由于LLMs训练数据来自大量开源代码，常生成不符合最佳实践甚至包含测试异味（anti-patterns）的测试。为此，我们提出基于静态质量指标的强化学习方法（RLSQM），通过静态分析构建奖励模型，优化五个代码质量指标。实验显示，经强化学习优化的Codex模型在所有质量指标上均优于基础LLM，最高提升23%，且生成代码几乎100%语法正确。尽管训练模型为成本更低的Codex，其表现仍优于GPT-4。本研究揭示了强化学习提升测试生成质量的可行性，为提高自动化测试效率与可靠性提供了有效路径。数据可在https://doi.org/10.6084/m9.figshare.25983166获取。
原文摘要 · Abstract (English)
Software testing is a crucial but time-consuming aspect of software development, and recently, Large Language Models (LLMs) have gained popularity for automated test case generation. However, because LLMs are trained on vast amounts of open-source code, they often generate test cases that do not adhere to best practices and may even contain test smells (anti-patterns). To address this issue, we propose Reinforcement Learning from Static Quality Metrics (RLSQM), wherein we utilize Reinforcement Learning to generate high-quality unit tests based on static analysis-based quality metrics. First, we analyzed LLM-generated tests and show that LLMs frequently do generate undesirable test smells -- up to 37% of the time. Then, we implemented lightweight static analysis-based reward model and trained LLMs using this reward model to optimize for five code quality metrics. Our experimental results demonstrate that the RL-optimized Codex model consistently generated higher-quality test cases than the base LLM, improving quality metrics by up to 23%, and generated nearly 100% syntactically-correct code. RLSQM also outperformed GPT-4 on all code quality metrics, in spite of training a substantially cheaper Codex model. We provide insights into how reliably utilize RL to improve test generation quality and show that RLSQM is a significant step towards enhancing the overall efficiency and reliability of automated software testing. Our data are available at https://doi.org/10.6084/m9.figshare.25983166.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。