用高质量数据微调小模型，可让大模型生成更优漏洞利用代码。
Data-Centric Benchmarking of Exploit Generation in LLMs: Understanding the Impact of Fine-Tuning
- 构建多阶段预处理数据集，设计细粒度评估框架
- 80亿参数模型微调后漏洞生成质量提升超42.5%
- 适合关注安全应用的开发者与研究者
我们研究了基于CVE条件的漏洞利用代码生成任务，即模型在给定软件漏洞上下文时生成概念验证（PoC）代码。采用数据驱动方法，通过多阶段预处理构建高质量数据集，并引入基于大模型评判和细粒度评分标准的可扩展评估框架。在此统一设置下，我们在8项评估指标上对17个大型语言模型进行了基准测试，系统分析其零样本能力。结果表明，一个小型80亿参数开源模型在高质量数据上微调后，漏洞生成质量提升超过42.5%，结合简单的测试时拒绝策略，可媲美部分专有模型。研究强调了数据质量、结构化监督和评估设计在可靠漏洞生成中的关键作用，提示这些因素在适应大模型至网络安全任务时可能与模型规模同等重要。
原文摘要 · Abstract (English)
We study the task of CVE-conditioned exploit generation, where a model drafts proof-of-concept (PoC) exploits given software vulnerability context. We adopt a data-centric approach, constructing a high-quality dataset via multi-stage preprocessing and introducing a scalable evaluation framework with LLM-as-judge and fine-grained rubrics. Under this unified setup, we benchmark 17 large language models across 8 evaluation criteria, providing systematic insights into their zero-shot capabilities. We further show that a compact 8B open-weight model, when fine-tuned on curated data, achieves over 42.5% improvement in exploit quality and rivals some proprietary models when combined with simple test-time rejection strategies. Our results highlight the importance of data quality, structured supervision, and evaluation design for reliable exploit generation, suggesting that these factors can be as critical as model scale in adapting LLMs to cybersecurity tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。