用大模型提升自动驾驶系统模糊测试效率，显著增加漏洞触发概率。
SAFLITE: Fuzzing Autonomous Systems via Large Language Models
- 基于大模型预测测试用例的安全相关性，指导高效测试
- 平均提升93.1%的漏洞触发率，部分工具提升超200%
- 适用于无人机等自主系统安全测试，适合安全研究人员
模糊测试能有效发现软件漏洞，但在自动驾驶系统（AS）中因搜索空间和状态空间庞大而面临挑战。本文提出通用框架SaFliTe，利用大语言模型（LLM）结合测试目标与系统状态，评估测试用例的相关性。实验中将SaFliTe集成至四种针对无人机控制系统的模糊测试工具：PGFuzz、DeepHyperion-UAV、CAMBA和TUMB，使用GPT-3.5、Mistral-7B、Llama2-7B等模型，在阿鲁皮洛特、PX4及PX4-Avoidance系统上测试。结果显示，相比PGFuzz，SaFliTe使每轮模糊测试中触发漏洞的操作选择概率平均提升93.1%；集成后，DeepHyperion-UAV、CAMBA、TUMB生成导致系统违规的测试用例能力分别提升234.5%、33.3%、17.8%。评估基准来自无人机测试竞赛。
原文摘要 · Abstract (English)
Fuzz testing effectively uncovers software vulnerabilities; however, it faces challenges with Autonomous Systems (AS) due to their vast search spaces and complex state spaces, which reflect the unpredictability and complexity of real-world environments. This paper presents a universal framework aimed at improving the efficiency of fuzz testing for AS. At its core is SaFliTe, a predictive component that evaluates whether a test case meets predefined safety criteria. By leveraging the large language model (LLM) with information about the test objective and the AS state, SaFliTe assesses the relevance of each test case. We evaluated SaFliTe by instantiating it with various LLMs, including GPT-3.5, Mistral-7B, and Llama2-7B, and integrating it into four fuzz testing tools: PGFuzz, DeepHyperion-UAV, CAMBA, and TUMB. These tools are designed specifically for testing autonomous drone control systems, such as ArduPilot, PX4, and PX4-Avoidance. The experimental results demonstrate that, compared to PGFuzz, SaFliTe increased the likelihood of selecting operations that triggered bug occurrences in each fuzzing iteration by an average of 93.1\%. Additionally, after integrating SaFliTe, the ability of DeepHyperion-UAV, CAMBA, and TUMB to generate test cases that caused system violations increased by 234.5\%, 33.3\%, and 17.8\%, respectively. The benchmark for this evaluation was sourced from a UAV Testing Competition.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。