用大模型驱动多智能体系统，自动完成REST API测试全流程。
MASTEST: A LLM-Based Multi-Agent System For RESTful API Tests
- 构建多智能体系统，融合大模型与程序化代理，实现从规范到脚本的自动化测试链。
- GPT-4o和DeepSeek在接口操作覆盖率达高，生成脚本语法正确率100%。
- 适合开发者快速生成高质量API测试用例，降低人工编写成本。
RESTful API测试在云原生应用质量保障中日益重要。本文提出MASTEST——一个基于大语言模型（LLM）的多智能体系统，将LLM与程序化代理结合，覆盖从OpenAPI Swagger规范生成单元与系统测试场景、生成Pytest测试脚本、执行脚本调用服务、分析响应结果判断测试正确性并计算覆盖率的完整流程。系统支持人工审核与修正生成的测试产物以保证质量。在GPT-4o与DeepSeek V3.1 Reasoner两个模型上，对五个公开API进行评估。测试覆盖度、数据类型正确性、状态码覆盖率、脚本语法正确性及缺陷检测能力等指标显示：两者整体表现优异；DeepSeek在数据类型与状态码识别上更优，GPT-4o在接口操作覆盖上领先；所有生成脚本均保持100%语法正确性，仅需少量人工修正语义即可使用。结果证明MASTEST的有效性与可行性。
原文摘要 · Abstract (English)
Testing RESTful API is increasingly important in quality assurance of cloud-native applications. Recent advances in machine learning (ML) techniques have demonstrated that various testing activities can be performed automatically by large language models (LLMs) with reasonable accuracy. This paper develops a multi-agent system called MASTEST that combines LLM-based and programmed agents to form a complete tool chain that covers the whole workflow of API test starting from generating unit and system test scenarios from API specification in the OpenAPI Swagger format, to generating of Pytest test scripts, executing test scripts to interact with web services, to analysing web service response messages to determine test correctness and calculate test coverage. The system also supports the incorporation of human testers in reviewing and correcting LLM generated test artefacts to ensure the quality of testing activities. MASTEST system is evaluated on two LLMs, GPT-4o and DeepSeek V3.1 Reasoner with five public APIs. The performances of LLMs on various testing activities are measured by a wide range of metrics, including unit and system test scenario coverage and API operation coverage for the quality of generated test scenarios, data type correctness, status code coverage and script syntax correctness for the quality of LLM generated test scripts, as well as bug detection ability and usability of LLM generated test scenarios and scripts. Experiment results demonstrated that both DeepSeek and GPT-4o achieved a high overall performance. DeepSeek excels in data type correctness and status code detection, while GPT-4o performs best in API operation coverage. For both models, LLM generated test scripts maintained 100\% syntax correctness and only required minimal manual edits for semantic correctness. These findings indicate the effectiveness and feasibility of MASTEST.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。