对比三种LLM代码漏洞检测方法，发现引入外部知识效果最佳。
An Empirical Evaluation of LLM-Based Approaches for Code Vulnerability Detection: RAG, SFT, and Dual-Agent Systems
- 用RAG融合互联网和MITRE数据库提升上下文理解
- RAG达到86%准确率和85%F1值，表现最优
- 双代理系统增强推理透明性，适合安全审计场景
大型语言模型（LLM）为自动化软件漏洞检测带来了新机遇。本文对比评估了三种基于LLM的方法：检索增强生成（RAG）、监督微调（SFT）和双代理框架。研究基于Big-Vul和GitHub真实仓库构建数据集，聚焦五个关键CWE类别：CWE-119、CWE-399、CWE-264、CWE-20和CWE-200。RAG方法通过整合互联网及MITRE CWE数据库的外部领域知识，实现最高整体准确率（0.86）和F1分数（0.85），凸显上下文增强的价值。SFT方法采用参数高效的QLoRA适配器，表现良好。双代理系统通过第二代理审核并优化第一代理输出，在提升推理透明性和错误缓解方面展现潜力，且资源开销更低。结果表明，融入领域知识机制显著增强LLM在实际漏洞检测中的可用性。
原文摘要 · Abstract (English)
The rapid advancement of Large Language Models (LLMs) presents new opportunities for automated software vulnerability detection, a crucial task in securing modern codebases. This paper presents a comparative study on the effectiveness of LLM-based techniques for detecting software vulnerabilities. The study evaluates three approaches, Retrieval-Augmented Generation (RAG), Supervised Fine-Tuning (SFT), and a Dual-Agent LLM framework, against a baseline LLM model. A curated dataset was compiled from Big-Vul and real-world code repositories from GitHub, focusing on five critical Common Weakness Enumeration (CWE) categories: CWE-119, CWE-399, CWE-264, CWE-20, and CWE-200. Our RAG approach, which integrated external domain knowledge from the internet and the MITRE CWE database, achieved the highest overall accuracy (0.86) and F1 score (0.85), highlighting the value of contextual augmentation. Our SFT approach, implemented using parameter-efficient QLoRA adapters, also demonstrated strong performance. Our Dual-Agent system, an architecture in which a secondary agent audits and refines the output of the first, showed promise in improving reasoning transparency and error mitigation, with reduced resource overhead. These results emphasize that incorporating a domain expertise mechanism significantly strengthens the practical applicability of LLMs in real-world vulnerability detection tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。