用ChatGPT从需求文档自动生成测试用例，有效提升测试效率与质量。
System Test Case Design from Requirements Specifications: Insights and Challenges of Using ChatGPT
- 通过提示链技术，将需求文档逐步转化为测试用例。
- 87%生成用例有效，15%为开发团队未考虑的新用例。
- 可辅助发现冗余用例，帮助优化测试套件。
系统测试在软件开发中至关重要，确保最终产品满足需求。从需求规格说明书中生成全面的测试用例通常困难且耗时。本文研究了使用大语言模型（LLMs）从软件需求规格说明书（SRS）生成测试用例设计的有效性。我们收集了五个软件工程项目的SRS文档，包含功能与非功能需求，这些项目由各自开发团队实现、测试并交付。采用ChatGPT-4o Turbo模型，通过提示链策略，先设置上下文，再为每个用例生成测试用例。测试用例质量由原开发团队反馈评估。实验表明，约87%的生成用例有效，剩余13%不适用或冗余；其中15%的有效用例是开发团队此前未考虑的。我们还让ChatGPT识别冗余用例，由开发团队验证，确认了误报并发现部分被忽略的冗余项。研究证明，利用LLMs从需求文档生成测试用例具有潜力，并能协助开发者快速识别与处理冗余，从而提升测试套件质量与测试效率。
原文摘要 · Abstract (English)
System testing is essential in any software development project to ensure that the final products meet the requirements. Creating comprehensive test cases for system testing from requirements is often challenging and time-consuming. This paper explores the effectiveness of using Large Language Models (LLMs) to generate test case designs from Software Requirements Specification (SRS) documents. In this study, we collected the SRS documents of five software engineering projects containing functional and non-functional requirements, which were implemented, tested, and delivered by respective developer teams. For generating test case designs, we used ChatGPT-4o Turbo model. We employed prompt-chaining, starting with an initial context-setting prompt, followed by prompts to generate test cases for each use case. We assessed the quality of the generated test case designs through feedback from the same developer teams as mentioned above. Our experiments show that about 87 percent of the generated test cases were valid, with the remaining 13 percent either not applicable or redundant. Notably, 15 percent of the valid test cases were previously not considered by developers in their testing. We also tasked ChatGPT with identifying redundant test cases, which were subsequently validated by the respective developers to identify false positives and to uncover any redundant test cases that may have been missed by the developers themselves. This study highlights the potential of leveraging LLMs for test generation from the Requirements Specification document and also for assisting developers in quickly identifying and addressing redundancies, ultimately improving test suite quality and efficiency of the testing procedure.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。