用大模型生成测试用例并保持多样性,提升自动化测试覆盖率。
EvoGPT: Leveraging LLM-Driven Seed Diversity to Improve Search-Based Test Suite Generation
- 用大模型生成初始测试集,通过多种温度与提示词保证多样性。
- 相比纯大模型或传统方法,代码覆盖率和变异得分平均提升10%。
- 适合想用AI提升测试质量的开发人员和测试工程师。
基于搜索的软件测试(SBST)在自动生成单元测试方面已广泛应用,但常面临过早收敛和测试集多样性不足的问题。最近,大语言模型(LLMs)成为单元测试生成的新方法。本文提出EvoGPT,一种将基于大模型的测试生成与基于SBST的测试套件优化相结合的混合系统。EvoGPT利用大模型生成初始测试集,并采用进化算法(EA)进一步优化该测试集。其关键创新在于显式强化多样性,通过生成时使用多种温度参数和不同提示指令实现。此外,每个大模型生成的测试用例均经过生成-修复循环及覆盖引导的断言生成进行精炼。为应对进化停滞,EvoGPT能检测搜索瓶颈,并注入针对未覆盖分支的大模型生成测试,同样通过多温度与提示指令保障多样性。在Defects4J标准基准上的评估显示,相比TestART(纯大模型基线)和EvoSuite(标准SBST基线),EvoGPT在代码覆盖率和变异得分上平均提升10%。消融实验表明,在初始化和搜索过程中显式强制多样性是有效利用大模型进行自动化测试生成的关键。
原文摘要 · Abstract (English)
Search-Based Software Testing (SBST) is a well-established approach for automated unit test generation, yet it often suffers from premature convergence and limited diversity in the generated test suites. Recently, Large Language Models (LLMs) have emerged as an alternative technique for unit test generation. We present EvoGPT, a hybrid test generation system that integrates LLM-based test generation with SBST-based test suite optimization. EvoGPT uses LLMs to generate an initial population of test suites, and uses an Evolutionary Algorithm (EA) to further optimize this test suite population. A distinguishing feature of EvoGPT is its explicit enforcement of diversity, achieved through the use of multiple temperatures and prompt instructions during test generation. In addition, each LLM-generated test is refined using a generation-repair loop and coverage-guided assertion generation. To address evolutionary plateaus, EvoGPT also detects stagnation during search and injects additional LLM-generated tests aimed at previously uncovered branches. Here too diversity is enforced using multiple temperatures and prompt instructions. We evaluate EvoGPT on Defects4J, a standard benchmark for test generation. The results show that EvoGPT achieves, on average, a 10% improvement in both code coverage and mutation score metrics compared to TestART, an LLM-only baseline; and EvoSuite, a standard SBST baseline. An ablation study indicates that explicitly enforcing diversity both at initialization and during the search is key to effectively leveraging LLMs for automated unit test generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。