用私有GPT根据需求自动生成可运行的测试代码，提升测试质量。
Private GPTs for LLM-driven testing in software development and machine learning
- 输入需求文本，分两步生成测试：先转Gherkin语法，再生成代码。
- 相比直接生成，两步法输出更易读、代码更规范，少用冗余库。
- 结构化提示词在简单程序和分类模型测试中均表现更优。
本文研究私有GPT基于需求自动生成可执行测试代码的能力。以现代开发流程中的史诗或用户故事中的验收标准为输入，使产品负责人或业务分析师可通过大语言模型直接生成可测试的标准。通过两种方式评估生成测试的质量：一是直接从需求生成代码，二是经由Gherkin语法作为中间步骤。结果表明，两步法在人类可读性和编码规范性方面更优，表现为代码行数更少、对测试常用库的依赖更合理。具体在‘Hello World’程序和数字分类模型两个场景下验证，结构化提示词显著提升测试输出质量。
原文摘要 · Abstract (English)
In this contribution, we examine the capability of private GPTs to automatically generate executable test code based on requirements. More specifically, we use acceptance criteria as input, formulated as part of epics, or stories, which are typically used in modern development processes. This gives product owners, or business intelligence, respectively, a way to directly produce testable criteria through the use of LLMs. We explore the quality of the so-produced tests in two ways: i) directly by letting the LLM generate code from requirements, ii) through an intermediate step using Gherkin syntax. As a result, it turns out that the two-step procedure yields better results -where we define better in terms of human readability and best coding practices, i.e. lines of code and use of additional libraries typically used in testing. Concretely, we evaluate prompt effectiveness across two scenarios: a simple "Hello World" program and a digit classification model, showing that structured prompts lead to higher-quality test outputs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。