对比多种机器学习方法在网络安全数据不平衡问题下的表现。
Cyber Security Data Science: Machine Learning Methods and their Performance on Imbalanced Datasets
- 测试六种单分类器与四种样本平衡技术的效果
- 不同数据集最佳模型各异,无通用最优解
- 建议针对新数据集重新评估模型与平衡策略
网络安全已成为全球范围内个人、机构和政府共同关注的议题。由于安全威胁持续不断,自动化检测成为必要手段。许多网络安全任务可建模为二分类问题,如异常检测、欺诈检测、入侵检测、垃圾邮件检测和恶意软件检测。本文开展三项实验:第一项评估随机森林、轻量梯度提升机、极端梯度提升、逻辑回归、决策树和梯度提升决策树六种单分类器;第二项测试过采样、欠采样、合成少数类过采样技术(SMOTE)及自适应步进集成(Self-Paced Ensembling)等采样方法;第三项评估自适应步进集成及其基分类器数量的影响。结果表明,不平衡学习技术既有正向作用也有潜在负面影响,需谨慎使用。此外,不同数据集表现最佳的模型各不相同,因此建议针对每个新数据集和应用场景,重新测试单分类器与不平衡学习技术。
原文摘要 · Abstract (English)
Cybersecurity has become essential worldwide and at all levels, concerning individuals, institutions, and governments. A basic principle in cybersecurity is to be always alert. Therefore, automation is imperative in processes where the volume of daily operations is large. Several cybersecurity applications can be addressed as binary classification problems, including anomaly detection, fraud detection, intrusion detection, spam detection, or malware detection. We present three experiments. In the first experiment, we evaluate single classifiers including Random Forests, Light Gradient Boosting Machine, eXtreme Gradient Boosting, Logistic Regression, Decision Tree, and Gradient Boosting Decision Tree. In the second experiment, we test different sampling techniques including over-sampling, under-sampling, Synthetic Minority Over-sampling Technique, and Self-Paced Ensembling. In the last experiment, we evaluate Self-Paced Ensembling and its number of base classifiers. We found that imbalance learning techniques had positive and negative effects, as reported in related studies. Thus, these techniques should be applied with caution. Besides, we found different best performers for each dataset. Therefore, we recommend testing single classifiers and imbalance learning techniques for each new dataset and application involving imbalanced datasets as is the case in several cyber security applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。