检验主流图神经网络入侵检测系统的真实效果与可复现性
Are We There Yet? Unraveling the State-of-the-Art Graph Network Intrusion Detection Systems
- 系统评估当前最先进图网络入侵检测模型的实现与性能
- 发现不同数据集上表现差异大，复现困难且误报率不稳定
- 适合关注安全模型可靠性与实验可复现性的研究者
网络入侵检测系统（NIDS）对保障企业安全至关重要。近年来，基于图的NIDS（GIDS）因其能有效捕捉通信数据中复杂的图结构关系而受到广泛关注。然而，这些GIDS的可复现性与可重复性尚未得到充分探索，影响了可靠检测系统的开发。本研究设计了一套系统性方法，对现有最先进的GIDS进行批判性评估、扩展和澄清。我们在三个公开数据集及一个新收集的大规模企业数据集上进行了评估。结果揭示显著的性能差异，凸显数据集规模、模型输入与实现设置带来的挑战。我们证实了复现与重复实验的困难，特别是在误报率和对抗攻击下的鲁棒性方面。该工作为未来研究提供了重要见解与建议，强调在构建稳健、通用的GIDS解决方案中，严谨的复现与重复研究的重要性。
原文摘要 · Abstract (English)
Network Intrusion Detection Systems (NIDS) are vital for ensuring enterprise security. Recently, Graph-based NIDS (GIDS) have attracted considerable attention because of their capability to effectively capture the complex relationships within the graph structures of data communications. Despite their promise, the reproducibility and replicability of these GIDS remain largely unexplored, posing challenges for developing reliable and robust detection systems. This study bridges this gap by designing a systematic approach to evaluate state-of-the-art GIDS, which includes critically assessing, extending, and clarifying the findings of these systems. We further assess the robustness of GIDS under adversarial attacks. Evaluations were conducted on three public datasets as well as a newly collected large-scale enterprise dataset. Our findings reveal significant performance discrepancies, highlighting challenges related to dataset scale, model inputs, and implementation settings. We demonstrate difficulties in reproducing and replicating results, particularly concerning false positive rates and robustness against adversarial attacks. This work provides valuable insights and recommendations for future research, emphasizing the importance of rigorous reproduction and replication studies in developing robust and generalizable GIDS solutions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。