用多个大模型协作检测修复漏洞，效果优于单个模型。
Diverse LLMs vs. Vulnerabilities: Who Detects and Fixes Them Better?
- 集成多个不同大模型的输出，提升漏洞检测与修复能力。
- 复杂代码下检测准确率提升10%-12%，多文件漏洞召回率增18%。
- 适合安全团队在不同风险场景下调整模型共识阈值使用。
大型语言模型（LLMs）正被广泛研究用于软件漏洞检测（SVD）和修复（SVR）。尽管单个LLM已具备代码理解能力，但在识别复杂漏洞和生成修复方案时仍表现不佳。本研究提出DVDR-LLM，一种融合多个异构LLM输出的集成框架，评估多模型聚合是否可降低错误率。实验表明，相较于单个模型平均性能，DVDR-LLM在检测准确率上提升10%-12%，且随着代码复杂度增加，优势更明显。对于多文件漏洞，该方法在召回率上提升18%，F1分数提高11.8%。然而，该方法带来权衡：在验证任务中降低误报，但检测任务中增加漏报，需根据安全场景谨慎设定模型间一致性的阈值以实现最优表现。代码与数据集详见：https://github.com/Erroristotle/DVDR_LLM。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are increasingly being studied for Software Vulnerability Detection (SVD) and Repair (SVR). Individual LLMs have demonstrated code understanding abilities, but they frequently struggle when identifying complex vulnerabilities and generating fixes. This study presents DVDR-LLM, an ensemble framework that combines outputs from diverse LLMs to determine whether aggregating multiple models reduces error rates. Our evaluation reveals that DVDR-LLM achieves 10-12% higher detection accuracy compared to the average performance of individual models, with benefits increasing as code complexity grows. For multi-file vulnerabilities, the ensemble approach demonstrates significant improvements in recall (+18%) and F1 score (+11.8%) over individual models. However, the approach raises measurable trade-offs: reducing false positives in verification tasks while simultaneously increasing false negatives in detection tasks, requiring careful decision on the required level of agreement among the LLMs (threshold) for increased performance across different security contexts. Artifact: https://github.com/Erroristotle/DVDR_LLM
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。