通过学习高阶策略提升漏洞复现成功率,让AI更智能地找到漏洞利用方法。
Mastermind: Strategy-grounded Learning for Repository-Scale Vulnerability Reproduction
- 分离策略学习与具体执行,用可复用的策略指导漏洞复现
- 在CyberGym上达84.5%成功率,显著优于传统方法
- 策略可跨模型迁移,提升多个大模型的漏洞复现能力
仓库级漏洞复现是软件工程中一项复杂任务:代理需分析代码库、推断触发漏洞路径的输入语法、构造原型证明(PoC),并验证补丁版本是否不再崩溃。尽管当前大语言模型代理在正确路径下可完成这些步骤,但仍因选择错误策略而失败。本文提出,策略而非完整动作轨迹,才是此类软件工程代理应学习的合适单元——它足够紧凑以优化,足够具体以引导执行,且足够稳定可重复使用。我们提出Mastermind,一种双循环框架,将可迁移的策略学习与任务特定经验分离。一个可训练规划器通过监督微调和基于里程碑的GRPO学习通用漏洞复现策略;同时,经验环维护局部策略记录以指导后续尝试。规划器与执行器独立训练,使策略学习可提升多个冻结的执行器,无需修改其生成动作的能力。我们在CyberGym上评估,使用260个训练任务和200个保留测试任务。以GPT-5.5为冻结执行器时,Mastermind达到84.5%通过率,显著优于开卷上下文(60.0%)、Best-of-8采样(63.0%)和迭代改进(77.0%)。同一规划器还将GPT-5.4 mini和GLM~5.1的性能从45.0%和58.5%提升至60.0%和71.0%。结果表明,学习高层策略是提升仓库级软件工程代理的有效且可迁移机制。
原文摘要 · Abstract (English)
Repository-level vulnerability reproduction is a demanding software engineering (SE) task: an agent must inspect a codebase, infer the input grammar that reaches a vulnerable path, construct a proof-of-conceptv(PoC), and verify that the crash disappears on the patched build. Recent LLM agents can often execute these steps when the approach is correct, yet they still fail by choosing the wrong strategy. This paper argues that strategy, rather than the full action trajectory, is the right learning unit for such SE agents: it is compact enough to optimize, concrete enough to guide execution, and stable enough to store and reuse across attempts. We present Mastermind, a dual-loop framework that separates transferable strategy learning from task-specific experience. A trainable planner learns reusable vulnerability-reproduction strategies through SFT and milestone-based GRPO, while an experience loop maintains task-local strategy records that guide subsequent attempts. The planner is trained independently of the executor, allowing strategy learning to improve multiple frozen executors without modifying their action-generation capability. We evaluate Mastermind on CyberGym using 260 training tasks and 200 held-out evaluation tasks. With GPT-5.5 as the frozen executor, Mastermind achieves an 84.5% pass rate, outperforming open-book PoC context (60.0%), Best-of-8 sampling (63.0%), and iterative improvement (77.0%). The same planner also improves GPT-5.4 mini and GLM~5.1 from 45.0% and 58.5% to 60.0% and 71.0%. These results demonstrate that learning high-level strategies is an effective and transferable mechanism for improving repository-scale SE agents.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。