揭示漏洞检测数据集三大核心问题，助模型更准识别软件漏洞。
Data Quality Issues in Vulnerability Detection Datasets
- 分析漏洞检测数据集的三类关键缺陷：样本不平衡、漏洞覆盖不足、分布偏倚。
- 实证14篇论文54个数据集，证实上述问题显著降低模型准确率。
- 提出数据清洗与构建建议，适合安全研究者和数据工程师参考。
漏洞检测是识别软件潜在弱点的关键任务，对网络安全至关重要。近年来，深度学习在自动化检测中取得显著进展，但其依赖大量标注源代码（有漏洞/无漏洞）来学习区分能力。现有文献中虽已创建多个数据集，但仍存在严重质量问题，影响模型性能。本文定义了三类主要问题：数据不平衡、漏洞覆盖率低、漏洞分布偏倚；以及三类次要问题：源码错误、标签错误、历史数据噪声。通过分析14篇论文及54个数据集验证了这些问题的存在。同时，文章提出使用和构建数据集的良好实践，以提升模型可靠性。
原文摘要 · Abstract (English)
Vulnerability detection is a crucial yet challenging task to identify potential weaknesses in software for cyber security. Recently, deep learning (DL) has made great progress in automating the detection process. Due to the complex multi-layer structure and a large number of parameters, a DL model requires massive labeled (vulnerable or secure) source code to gain knowledge to effectively distinguish between vulnerable and secure code. In the literature, many datasets have been created to train DL models for this purpose. However, these datasets suffer from several issues that will lead to low detection accuracy of DL models. In this paper, we define three critical issues (i.e., data imbalance, low vulnerability coverage, biased vulnerability distribution) that can significantly affect the model performance and three secondary issues (i.e., errors in source code, mislabeling, noisy historical data) that also affect the performance but can be addressed through a dedicated pre-processing procedure. In addition, we conduct a study of 14 papers along with 54 datasets for vulnerability detection to confirm these defined issues. Furthermore, we discuss good practices to use existing datasets and to create new ones.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。