用大模型代理自动检测并验证Node.js包中的漏洞,准确率超传统工具四倍。
Taint-Style Vulnerability Detection and Confirmation for Node.js Packages Using LLM Agent Reasoning
- 构建多阶段大模型代理流水线,结合代码扫描与可执行验证
- 在公开基准上确认84%漏洞,远超传统工具不足22%的准确率
- 无需标注数据或历史漏洞报告,适合新发布包的快速安全审查
快速发展的Node.js生态包含数百万个包,是现代软件供应链的关键部分,因此对其中漏洞的检测日益重要。然而,传统程序分析因动态JavaScript特性及庞大依赖关系而面临挑战。近年来,大语言模型(LLMs)及基于LLM的智能体范式为替代人工编写的程序模型提供了新思路。本文探讨了以大模型为核心、工具增强的方法是否能有效检测并确认Node.js包中的污染型漏洞(如任意命令注入)。我们实现了LLMVD.js,一个分阶段的智能体流水线,用于扫描代码、提出漏洞、生成概念验证攻击,并通过轻量级执行断言进行验证;并在无专用静态/动态分析引擎支持路径推导的情况下,系统评估其在污染型漏洞检测与确认上的效果。在公共基准数据集上,LLMVD.js确认了84%的漏洞,而此前程序分析工具的确认率不足22%。它还优于先前的混合式方法,且无需漏洞标注或历史漏洞报告。在260个新发布的包(无漏洞真实标签)上的评估中,传统工具仅能验证≤2个包的漏洞,而LLMVD.js成功生成了36个可验证的漏洞利用。
原文摘要 · Abstract (English)
The rapidly evolving Node$.$js ecosystem currently includes millions of packages and is a critical part of modern software supply chains, making vulnerability detection of Node$.$js packages increasingly important. However, traditional program analysis struggles in this setting because of dynamic JavaScript features and the large number of package dependencies. Recent advances in large language models (LLMs) and the emerging paradigm of LLM-based agents offer an alternative to handcrafted program models. This raises the question of whether an LLM-centric, tool-augmented approach can effectively detect and confirm taint-style vulnerabilities (e.g., arbitrary command injection) in Node$.$js packages. We implement LLMVD$.$js, a multi-stage agent pipeline to scan code, propose vulnerabilities, generate proof-of-concept exploits, and validate them through lightweight execution oracles; and systematically evaluate its effectiveness in taint-style vulnerability detection and confirmation in Node$.$js packages without dedicated static/dynamic analysis engines for path derivation. For packages from public benchmarks, LLMVD$.$js confirms 84% of the vulnerabilities, compared to less than 22% for prior program analysis tools. It also outperforms a prior LLM-program-analysis hybrid approach while requiring neither vulnerability annotations nor prior vulnerability reports. When evaluated on a set of 260 recently released packages (without vulnerability groundtruth information), traditional tools produce validated exploits for few ($\leq 2$) packages, while LLMVD$.$js generates validated exploits for 36 packages.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。