用强化学习让小模型生成复杂文本输入,高效发现软件漏洞。
R1-Fuzz: Specializing Language Models for Textual Fuzzing via Reinforcement Learning
- 用强化学习微调小语言模型,结合语法语义约束生成测试用例。
- 在真实系统上实现75%更高覆盖率,发现29个未知漏洞。
- 适合安全研究者、自动化测试团队使用,尤其针对复杂程序。
模糊测试能有效发现漏洞,但在编译器、解释器和数据库引擎等复杂目标上受限于文本输入需满足复杂的语法和语义约束。尽管语言模型因具备丰富的潜在知识与推理能力而受到关注,但其实际应用仍受制于对真实代码库深层程序逻辑探索不足及大模型成本过高。为此,我们提出R1-Fuzz,首个利用强化学习(RL)定制低成本语言模型并集成于复杂文本模糊测试输入生成的框架。R1-Fuzz引入两项关键设计:基于覆盖切片的问题构建方法与基于距离的奖励计算机制。通过在自建数据集上进行强化学习后训练,该框架使小型模型在模糊测试中能够深入推理程序语义。在多种真实目标上的评估表明,名为R1-Fuzz-7B的小模型可媲美甚至超越更大模型的表现。显著地,R1-Fuzz相较现有最先进模糊测试工具实现最高75%的覆盖率提升,并发现了29个此前未知的漏洞,验证了其实用性。
原文摘要 · Abstract (English)
Fuzzing is effective for vulnerability discovery but struggles with complex targets such as compilers, interpreters, and database engines, which accept textual input that must satisfy intricate syntactic and semantic constraints. Although language models (LMs) have attracted interest for this task due to their vast latent knowledge and reasoning potential, their practical adoption has been limited. The major challenges stem from insufficient exploration of deep program logic among real-world codebases, and the high cost of leveraging larger models. To overcome these challenges, we propose R1-Fuzz, the first framework that leverages reinforcement learning (RL) to specialize cost-efficient LMs and integrate them for complex textual fuzzing input generation. R1-Fuzz introduces two key designs: coverage-slicing-based question construction and a distance-based reward calculation. Through RL-based post-training of a model with our constructed dataset, R1-Fuzz designs a fuzzing workflow that tightly integrates LMs to reason deep program semantics during fuzzing. Evaluations on diverse real-world targets show that our design enables a small model, named R1-Fuzz-7B, to rival or even outperform much larger models in real-world fuzzing. Notably, R1-Fuzz achieves up to 75\% higher coverage than state-of-the-art fuzzers and discovers 29 previously unknown vulnerabilities, demonstrating its practicality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。