arXiv:2503.04178cs.CRcs.LG2025-03

用流学习算法在BETH数据集上实现实时异常检测,识别未知网络威胁。

Unsupervised anomaly detection on cybersecurity data streams: a case with BETH dataset

  • 采用十种流机器学习算法处理实时网络安全数据流。
  • 在BETH数据集上实现ROC-AUC超过0.92,总处理时间低于15秒。
  • 适合需要低延迟威胁检测的工业安全系统部署。

随着信息技术基础设施的发展,信息安全工具生成的安全事件数量持续增长。网络威胁环境不断变化,监控需兼顾已知攻击特征与尚未有签名规则的新型威胁。在大规模网络安全数据流中检测异常是一项复杂任务,若有效解决,可及时响应未知攻击。离线算法因训练和重训耗时而受限,流学习算法则能实现近实时处理。本文评估了三个Python流学习库中的十种算法在BETH数据集上的表现,该数据集通过扩展eBPF收集操作系统进程创建、克隆和销毁信息。实验对比了多种特征组合与事件顺序,报告了各算法的ROC-AUC值及总处理时间。结果表明,部分算法在保持高检测率的同时处理效率优异,为未来研究方向提供了建议。

原文摘要 · Abstract (English)

In modern world the importance of cybersecurity of various systems is increasing from year to year. The number of information security events generated by information security tools grows up with the development of the IT infrastructure. At the same time, the cyber threat landscape does not remain constant, and monitoring should take into account both already known attack indicators and those for which there are no signature rules in information security products of various classes yet. Detecting anomalies in large cybersecurity data streams is a complex task that, if properly addressed, can allow for timely response to atypical and previously unknown cyber threats. The possibilities of using of offline algorithms may be limited for a number of reasons related to the time of training and the frequency of retraining. Using stream learning algorithms for solving this task is capable of providing near-real-time data processing. This article examines the results of ten algorithms from three Python stream machine-learning libraries on BETH dataset with cybersecurity events, which contains information about the creation, cloning, and destruction of operating system processes collected using extended eBPF. ROC-AUC metric and total processing time of processing with these algorithms are presented. Several combinations of features and the order of events are considered. In conclusion, some mentions are given about the most promising algorithms and possible directions for further research are outlined.

异常检测流学习网络安全

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。