用大模型推理提升零样本漏洞检测，效果显著优于传统方法。
Reasoning with LLMs for Zero-Shot Vulnerability Detection
- 设计结构化推理提示，让大模型分步思考并验证结果。
- Think & Verify策略使模糊回答率从20.3%降至9.1%，准确率提升。
- 专用代码模型表现更优，不同漏洞类型需针对性方法。
自动化软件漏洞检测（SVD）在日益复杂和相互依赖的软件系统时代仍面临重大挑战。尽管大语言模型（LLMs）在代码分析方面取得进展，但现有评估方法常缺乏捕捉真实世界复杂性和跨组件交互所需的上下文感知鲁棒性。为此，我们提出VulnSage：一个从多样化、大规模开源系统软件项目（使用C/C++开发）中构建的综合评估框架与数据集。不同于以往数据集，它结合启发式噪声预过滤与基于LLM的推理，确保漏洞样本具有代表性且噪声最小。该框架支持函数、文件及跨函数多粒度分析，并采用四种零样本提示策略：基线、思维链、思考与思考&验证。评估发现，结构化推理提示显著提升性能，其中思考&验证将模糊响应从20.3%降至9.1%，同时提高准确率。进一步表明，代码专用模型始终优于通用模型，且不同漏洞类型的表现差异明显，说明单一方法无法在所有安全场景下通用。
原文摘要 · Abstract (English)
Automating software vulnerability detection (SVD) remains a critical challenge in an era of increasingly complex and interdependent software systems. Despite significant advances in Large Language Models (LLMs) for code analysis, prevailing evaluation methodologies often lack the \textbf{context-aware robustness} necessary to capture real-world intricacies and cross-component interactions. To address these limitations, we present \textbf{VulnSage}, a comprehensive evaluation framework and a dataset curated from diverse, large-scale open-source system software projects developed in C/C++. Unlike prior datasets, it leverages a heuristic noise pre-filtering approach combined with LLM-based reasoning to ensure a representative and minimally noisy spectrum of vulnerabilities. The framework supports multi-granular analysis across function, file, and inter-function levels and employs four diverse zero-shot prompt strategies: Baseline, Chain-of-Thought, Think, and Think & Verify. Through this evaluation, we uncover that structured reasoning prompts substantially improve LLM performance, with Think & Verify reducing ambiguous responses from 20.3% to 9.1% while increasing accuracy. We further demonstrate that code-specialized models consistently outperform general-purpose alternatives, with performance varying significantly across vulnerability types, revealing that no single approach universally excels across all security contexts. Link to dataset and codes: https://github.com/Erroristotle/VulnSage.git
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。