对比九种AutoML框架在不平衡网络入侵检测中的表现,发现集成与平衡优化是关键。
Evaluation of AutoML Frameworks for IDS under Imbalanced Data Conditions of the NSL-KDD Dataset
- 采用统一实验协议,评估9个开源AutoML框架在真实五类不平衡数据上的表现。
- PyCaret以66%宏平均F1领先,缺乏平衡机制的框架对罕见攻击识别严重退化。
- 强调仅优化准确率不足,需内置平衡策略和分层评估,适合安全研究者参考。
本研究探讨了严重类别不平衡对多类网络入侵检测中自动化机器学习(AutoML)框架性能的影响,使用NSL-KDD数据集进行评估。与以往简化为二分类或移除少数类的研究不同,本文保留原始五类分布,包含高度稀疏的R2L和U2R攻击,实现对不平衡敏感学习行为的真实评估。在统一可复现的实验协议下,分析了九个开源AutoML框架,涵盖架构设计、集成策略、验证流程、超参数优化及不平衡处理机制的差异。结果表明,结合集成学习与不平衡感知优化的框架在少数类识别上表现更优。PyCaret取得最佳整体性能,宏平均F1达66%,其次为AutoGluon(55%),而缺乏原生平衡支持的框架在少数类检测上显著退化。分析还显示,仅以准确率为优化目标不足以应对高度不平衡场景,因高权重指标可能伴随对罕见攻击类别的泛化能力差。本研究建立了基于AutoML的入侵检测在严重多类不平衡下的标准化基准,揭示当前架构局限性,强调需将不平衡感知优化、重采样与分层评估原生集成至自动化学习流水线。源代码已公开。
原文摘要 · Abstract (English)
This work investigates the impact of severe class imbalance on the performance of automated machine learning (AutoML) frameworks for multiclass network intrusion detection using the NSL-KDD dataset. Unlike previous studies that simplify the problem through binary classification or minority-class removal, we preserve the original five-class distribution, including highly underrepresented attacks such as R2L and U2R, enabling a realistic evaluation of imbalance-sensitive learning behavior. Nine open-source AutoML frameworks were analyzed under a unified and reproducible experimental protocol, considering differences in architectural design, ensemble strategies, validation procedures, hyperparameter optimization, and imbalance-handling mechanisms. The results demonstrate that frameworks incorporating ensemble learning and imbalance-aware optimization achieve better minority-class discrimination. PyCaret obtained the best overall performance, reaching 66\% macro-F1, followed by AutoGluon with 55\%, whereas frameworks lacking native balancing support exhibited significant degradation in minority-class detection capability. The analysis further shows that accuracy-oriented optimization alone is insufficient for highly imbalanced IDS scenarios, since high-weighted metrics may coexist with poor generalization on rare attack categories. As a contribution, this work establishes a standardized benchmark for AutoML-based intrusion detection under severe multiclass imbalance, highlighting current architectural limitations and the need for native integration of imbalance-aware optimization, resampling, and stratified evaluation strategies into automated learning pipelines. The source code is publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。