arXiv:2506.16899cs.CRcs.AI2025-06被引 5

用大模型自动识别代码安全检测中的误报,提升分析效率。

Towards Effective Complementary Security Analysis using Large Language Models

  • 用链式思考等提示技术让大模型判断代码漏洞报告真伪。
  • 在基准数据集上识别出约62.5%的误报,且未漏掉真实漏洞。
  • 多模型协作可将误报识别率提升至78.9%,适合安全团队使用。

安全分析中的一个关键挑战是人工评估静态应用安全测试(SAST)工具生成的安全弱点报告。这些报告中存在大量误报(FPs),降低了分析效率。本文提出利用大语言模型(LLMs)改进SAST结果的评估。基于从OWASP Benchmark(v1.2)和一个真实项目中提取的数据集,研究发现先进提示技术(如链式思考、自一致)显著提升了误报检测能力。值得注意的是,部分LLMs在不遗漏真实漏洞的前提下,识别出约62.5%的误报;若结合多个LLM的判断,则误报识别率可提升至约78.9%。此外,我们在涵盖五种SAST工具、三种编程语言及基础设施文件的真实数据集上验证了方法的泛化能力:最优LLM识别出33.85%的误报,多模型协同可达到38.46%。结果表明,LLMs能有效补充传统SAST工具,提升自动化水平并减少处理误报的资源消耗。

原文摘要 · Abstract (English)

A key challenge in security analysis is the manual evaluation of potential security weaknesses generated by static application security testing (SAST) tools. Numerous false positives (FPs) in these reports reduce the effectiveness of security analysis. We propose using Large Language Models (LLMs) to improve the assessment of SAST findings. We investigate the ability of LLMs to reduce FPs while trying to maintain a perfect true positive rate, using datasets extracted from the OWASP Benchmark (v1.2) and a real-world software project. Our results indicate that advanced prompting techniques, such as Chain-of-Thought and Self-Consistency, substantially improve FP detection. Notably, some LLMs identified approximately 62.5% of FPs in the OWASP Benchmark dataset without missing genuine weaknesses. Combining detections from different LLMs would increase this FP detection to approximately 78.9%. Additionally, we demonstrate our approach's generalizability using a real-world dataset covering five SAST tools, three programming languages, and infrastructure files. The best LLM detected 33.85% of all FPs without missing genuine weaknesses, while combining detections from different LLMs would increase this detection to 38.46%. Our findings highlight the potential of LLMs to complement traditional SAST tools, enhancing automation and reducing resources spent addressing false alarms.

大模型安全分析误报检测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。