用AI自动找漏洞测试用例,解决安全测试用例稀缺问题
A Match Made in Heaven? AI-driven Matching of Vulnerabilities and Security Unit Tests
- 用UniXcoder识别安全相关测试用例
- 用DeepSeek Coder匹配测试与漏洞,准确率达75%
- 已在427个开源项目中验证,构建新数据集Test4Vul
软件漏洞常通过污点分析、渗透测试或模糊测试发现,也可通过特定输入触发安全敏感行为的单元测试(即漏洞见证测试)发现。生成式AI可辅助编写此类测试,但需大量示例,而现有样本稀缺。本文提出VuTeCo框架,从Java仓库中收集漏洞见证测试实例。该框架完成两项任务:(1) “发现”任务——判断单元测试是否与安全相关;(2) “匹配”任务——将测试用例与所见证的漏洞关联。在Vul4J数据集上,
原文摘要 · Abstract (English)
Software vulnerabilities are often detected via taint analysis, penetration testing, or fuzzing. They are also found via unit tests that exercise security-sensitive behavior with specific inputs, called vulnerability-witnessing tests. Generative AI models could help developers in writing them, but they require many examples to learn from, which are currently scarce. This paper introduces VuTeCo, an AI-driven framework for collecting examples of vulnerability-witnessing tests from Java repositories. VuTeCo carries out two tasks: (1) The "Finding" task to determine whether a unit test case is security-related, and (2) the "Matching" task to relate a test case to the vulnerability it witnesses. VuTeCo addresses the Finding task with UniXcoder, achieving an F0.5 score of 0.73 and a precision of 0.83 on a test set of unit tests from Vul4J. The Matching task is addressed using DeepSeek Coder, achieving an F0.5 score of 0.65 and a precision of 0.75 on a test set of pairs of unit tests and vulnerabilities from Vul4J. VuTeCo has been used in the wild on 427 Java projects and 1,238 vulnerabilities, obtaining 224 test cases confirmed to be security-related and 35 tests correctly matched to 29 vulnerabilities. The validated tests were collected in a new dataset called Test4Vul. VuTeCo lays the foundation for large-scale retrieval of vulnerability-witnessing tests, enabling future AI models to better understand and generate security unit tests.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。