用检索增强生成+思维链，让大模型更精准生成协议测试用例。
Retrieval Augmented Generation Based LLM Evaluation For Protocol State Machine Inference With Chain-of-Thought Reasoning
- 结合RAG与思维链，动态调用RFC文档推理协议状态机。
- 生成用例在结构上比基线提升18.19%（BLEU）、23.45%（WER）。
- 适合做协议漏洞挖掘的自动化测试人员或安全研究者。
本文提出一种基于检索增强生成（RAG）的智能体大语言模型评估方法，用于网络数据包种子生成与增强。通过链式思维（COT）提示技术，提升种子的结构质量，以引导协议模糊测试框架更广泛地探索协议状态空间。该方法利用RAG与文本嵌入动态访问请求注释（RFC）知识库，回答关于协议有限状态机（FSM）的问题，并迭代推理所获知识，实现输出优化与种子合理布局。我们基于BLEU、ROUGE和词错误率（WER）等指标，对比生成包与真实包，评估代理输出结构质量。实验表明，相比基线模型，本方法在BLEU、ROUGE和WER上分别提升18.19%、14.81%和23.45%，验证了其在提升基于大模型的协议模糊测试框架发现隐藏漏洞能力方面的潜力。
原文摘要 · Abstract (English)
This paper presents a novel approach to evaluate the efficiency of a RAG-based agentic Large Language Model (LLM) architecture for network packet seed generation and enrichment. Enhanced by chain-of-thought (COT) prompting techniques, the proposed approach focuses on the improvement of the seeds' structural quality in order to guide protocol fuzzing frameworks through a wide exploration of the protocol state space. Our method leverages RAG and text embeddings to dynamically reference to the Request For Comments (RFC) documents knowledge base for answering queries regarding the protocol's Finite State Machine (FSM), then iteratively reasons through the retrieved knowledge, for output refinement and proper seed placement. We then evaluate the response structure quality of the agent's output, based on metrics as BLEU, ROUGE, and Word Error Rate (WER) by comparing the generated packets against the ground-truth packets. Our experiments demonstrate significant improvements of up to 18.19%, 14.81%, and 23.45% in BLEU, ROUGE, and WER, respectively, over baseline models. These results confirm the potential of such approach, improving LLM-based protocol fuzzing frameworks for the identification of hidden vulnerabilities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。