用AI自动生成测试用例，无须运行代码就能发现潜在错误。
Cerberus: Multi-Agent Reasoning and Coverage-Guided Exploration for Static Detection of Runtime Errors
- 利用大模型生成触发错误的输入并预测代码覆盖率
- 通过双阶段反馈机制，高效提升覆盖率并发现更多运行时错误
- 适合在代码合并前快速检测不完整代码的潜在缺陷
在软件开发中，常需在不实际执行代码的情况下检测运行时错误，例如在将在线代码片段集成到项目前进行审查。本文提出Cerberus，一种新型的、无需执行的覆盖引导式测试框架。该框架利用大语言模型（LLMs）生成能触发运行时错误的输入，并在不执行代码的前提下完成代码覆盖率预测与错误检测。Cerberus采用两阶段反馈循环：第一阶段同时提升代码覆盖率并检测错误；当覆盖率达到100%或上限后，第二阶段仅专注于错误检测，从而优于直接用大模型同时完成两项任务的方法。实证评估表明，相较于传统及基于学习的测试框架，Cerberus能更高效地生成高覆盖率测试用例，发现更多运行时错误，尤其适用于不完整代码片段的静态检测。
原文摘要 · Abstract (English)
In several software development scenarios, it is desirable to detect runtime errors and exceptions in code snippets without actual execution. A typical example is to detect runtime exceptions in online code snippets before integrating them into a codebase. In this paper, we propose Cerberus, a novel predictive, execution-free coverage-guided testing framework. Cerberus uses LLMs to generate the inputs that trigger runtime errors and to perform code coverage prediction and error detection without code execution. With a two-phase feedback loop, Cerberus first aims to both increasing code coverage and detecting runtime errors, then shifts to focus only detecting runtime errors when the coverage reaches 100% or its maximum, enabling it to perform better than prompting the LLMs for both purposes. Our empirical evaluation demonstrates that Cerberus performs better than conventional and learning-based testing frameworks for (in)complete code snippets by generating high-coverage test cases more efficiently, leading to the discovery of more runtime errors.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。