用大模型自动生成单元测试，关键在代码上下文和提示策略。
Impact of Code Context and Prompting Strategies on Automated Unit Test Generation with Modern General-Purpose Large Language Models
- 通过链式思维提示提升生成质量，效果优于普通提示。
- 加入函数文档字符串可显著提高测试覆盖度，全实现代码增益有限。
- M5（Gemini 2.5 Pro）表现最佳，分支覆盖率高达96.3%。
生成式AI在软件工程中日益受到关注，而测试是保障可靠性的重要手段。根据流行的测试金字塔理论，单元测试占多数且结构化强，几乎无需领域知识。在软件开发阶段由工程师监督下自动产生此类测试，能显著提升效率。本文研究了代码上下文与提示策略对多种大型语言模型（LLMs）生成单元测试的质量与完备性影响。结果显示，包含文档字符串可明显提升代码完备性，而扩展至完整实现代码带来的增益较小。特别地，链式思维提示策略——即使应用于‘推理型’模型——取得最优效果：最高达96.3%的分支覆盖率、平均57%的变异测试得分，以及近乎完美的编译成功率。在所评估模型中，M5（Gemini 2.5 Pro）在变异得分和分支覆盖率上均表现领先，编译成功率也位居前列。所有代码与生成的测试集已公开于 https://github.com/peetery/LLM-analysis。
原文摘要 · Abstract (English)
Generative AI is gaining increasing attention in software engineering, where testing remains an indispensable reliability mechanism. According to the widely adopted testing pyramid, unit tests constitute the majority of test cases and are often schematic, requiring minimal domain expertise. Automatically generating such tests under the supervision of software engineers can significantly enhance productivity during the development phase of the software lifecycle. This paper investigates the impact of code context and prompting strategies on the quality and adequacy of unit tests generated by various large language models (LLMs) across several families. The results show that including docstrings notably improves code adequacy, while further extending context to the full implementation yields definitely smaller gains. Notably, the chain-of-thought prompting strategy -- applied even to 'reasoning' models -- achieves the best results, with up to 96.3\% branch coverage, a 57\% average mutation score, and near-perfect compilation success rate. Among the evaluated models, M5 (Gemini 2.5 Pro) demonstrated superior performance in both mutation score and branch coverage being still in top in terms of compilation success rate. All the code and resulting test suites are publicly available at https://github.com/peetery/LLM-analysis.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。