用大模型多智能体自动生成API测试用例，解决输出难验证的问题。
Multi-Agent LLM-based Metamorphic Testing for REST APIs
- 构建多智能体流程，自动生成符合Given-When-Then格式的测试场景。
- 在两个公开Web应用上测试，发现传统方法遗漏的异常行为。
- 适合需要提升REST API测试覆盖率的开发者和测试工程师。
随着REST API在软件系统中日益重要，其验证变得愈发关键。然而，由于难以判断API调用输出是否正确（即测试预言机问题），测试面临挑战。元变换测试是一种基于规范的测试方法，适用于正确输出未知或未明确定义的情况，通过指定不同输出之间的关系来验证系统正确性。本文提出ARMeta，一种基于大模型的多智能体工作流，用于支持以OpenAPI文档化的REST API的元变换测试。该智能体流程可识别元变换测试场景，并将其以Given-When-Then格式规范表达。这些场景被自动实现为可执行测试，并在目标系统上运行。我们在两个公开可用的暴露REST接口的Web应用上评估ARMeta，与基于场景的测试基线进行对比。结果表明，ARMeta探索的行为可有效补充现有场景化测试方法，揭示潜在缺陷。
原文摘要 · Abstract (English)
As REST APIs become an increasingly significant part of software systems, their validation is becoming more critical. Hence, testing and uncovering underlying issues are of utmost importance for improving software quality. However, testing REST APIs is challenging mainly due to the difficulty of assessing whether the output of an API call is correct, i.e., the test oracle problem. Metamorphic testing is a specification-based testing approach for situations where correct outputs are unknown or not specified explicitly. To check the correctness of a system, relations between the different outputs are specified. We present ARMeta, a tool-supported approach that uses an LLM-based multi-agent workflow to support metamorphic testing of REST APIs documented with OpenAPI. The agentic workflow is used to identify metamorphic test scenarios and specify them in the Given-When-Then format. These scenarios are automatically implemented as executable tests and executed against the system under test. We evaluate ARMeta on two publicly available web applications that expose REST interfaces and compare its performance with a scenario-based testing baseline. The results show that ARMeta explores behaviors that serve as a complement to existing scenario-based testing approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。