用大模型自动测试网络协议,效率提升8.65倍
Automated Network Protocol Testing with LLM Agents

- 多智能体大模型分层理解协议规范,自动生成测试用例
- 生成4632个测试用例,覆盖41个历史漏洞,是标准方法的4倍
- 全流程自动化,适合网络协议开发者和测试工程师使用
网络协议测试是现代网络基础设施的基础。传统方法依赖人工解读规范、设计用例并转化为可执行代码,每条用例需约一天工时,易出错且效率低。现有基于模型的方法虽部分自动化,仍需大量人工建模与专家干预,成本高且难以适应多样化、快速演进的协议。本文提出首个端到端自动化系统NeTestLLM,利用多智能体大语言模型实现协议理解、迭代生成、任务专用工作流与运行时反馈分析。该系统已在生产环境部署数月,获领域专家好评。实验中,对OSPF、RIP和BGP协议生成4,632个测试用例,覆盖41个FRRouting历史漏洞,远超当前国家标准的11个;生成可执行代码的效率较人工方法提升8.65倍。NeTestLLM为异构网络协议提供了首个实用的大模型驱动自动化测试方案。
原文摘要 · Abstract (English)
Network protocol testing is fundamental for modern network infrastructure. However, traditional network protocol testing methods are labor-intensive and error-prone, requiring manual interpretation of specifications, test case design, and translation into executable artifacts, typically demanding one person-day of effort per test case. Existing model-based approaches provide partial automation but still involve substantial manual modeling and expert intervention, leading to high costs and limited adaptability to diverse and evolving protocols. In this paper, we propose a first-of-its-kind system called NeTestLLM that takes advantage of multi-agent Large Language Models (LLMs) for end-to-end automated network protocol testing. NeTestLLM employs hierarchical protocol understanding to capture complex specifications, iterative test case generation to improve coverage, a task-specific workflow for executable artifact generation, and runtime feedback analysis for debugging and refinement. NeTestLLM has been deployed in a production environment for several months, receiving positive feedback from domain experts. In experiments, NeTestLLM generated 4,632 test cases for OSPF, RIP, and BGP, covering 41 historical FRRouting bugs compared to 11 by current national standards. The process of generating executable artifacts also improves testing efficiency by a factor of 8.65x compared to manual methods. NeTestLLM provides the first practical LLM-powered solution for automated end-to-end testing of heterogeneous network protocols.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。