用提示词让大模型检测漏洞，效果优于传统静态分析工具。
Can LLM Prompting Serve as a Proxy for Static Analysis in Vulnerability Detection
- 设计结合自然语言描述与对比思维链的提示策略
- 在部分代码场景下准确率提升31.6%，误报率降低37.6%
- 适合想替代人工规则的漏洞检测研究者或开发者
尽管大语言模型（LLM）在诸多任务中表现优异，但在安全关键型代码任务如漏洞检测方面仍显不足。传统静态分析（SA）工具（如CodeQL、CodeGuru Security）依赖人工手工构建规则，其效果受限于专家能力，且在部分代码场景下常出现高误报率。本研究探索在部分代码设置下，是否可用LLM提示作为静态分析的替代方案。提出融合漏洞自然语言描述与对比思维链推理的提示策略，并利用合成数据集中的对比样本进行增强。实验表明，具备安全意识的提示技术可有效替代手工规则，显著降低漏报率。在使用DeepSeek-R1等先进推理模型时，最优策略使准确率提升31.6%，F1分数提高71.7%，成对准确率提升60.4%，漏报率最高下降37.6%。
原文摘要 · Abstract (English)
Despite their remarkable success, large language models (LLMs) have shown limited ability on safety-critical code tasks such as vulnerability detection. Typically, static analysis (SA) tools, like CodeQL, CodeGuru Security, etc., are used for vulnerability detection. SA relies on predefined, manually-crafted rules for flagging various vulnerabilities. Thus, effectiveness of SA in detecting vulnerabilities depends on human experts and is known to report high error rates. In this study we investigate whether LLM prompting can be an effective alternative to these static analyzers in the partial code setting. We propose prompting strategies that integrate natural language instructions of vulnerabilities with contrastive chain-of-thought reasoning, augmented using contrastive samples from a synthetic dataset. Our findings demonstrate that security-aware prompting techniques can be effective alternatives to the laborious, hand-crafted rules of static analyzers, which often result in high false negative rates in the partial code setting. When leveraging SOTA reasoning models such as DeepSeek-R1, each of our prompting strategies exceeds the static analyzer baseline, with the best strategies improving accuracy by as much as 31.6%, F1-scores by 71.7%, pairwise accuracies by 60.4%, and reducing FNR by as much as 37.6%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。