arXiv:2511.12743cs.CRcs.LG2025-11被引 7

用MITRE ATT&CK评估网络安全数据集,让模型更贴合真实行业威胁。

An Evaluation Framework for Network IDS/IPS Datasets: Leveraging MITRE ATT&CK and Industry Relevance Metrics

  • 结合威胁情报与自然语言处理,多维度评估数据集适用性。
  • 发现医疗、能源、金融等关键行业数据覆盖严重不足。
  • 提供可落地的选数据方法,适合安全系统研发人员使用。

机器学习与深度学习驱动的入侵检测与防御系统(IDS/IPS)性能高度依赖训练与评估数据的质量与相关性。然而,当前AI模型评估普遍仅关注准确率,忽视数据是否反映实际行业威胁。为此,本文提出一种融合MITRE ATT&CK知识库的多维评估框架,采用五项互补指标综合衡量数据集在特定行业场景下的适用性。该框架整合威胁情报、自然语言处理与量化分析方法,应用于九个公开可用的IDS/IPS数据集,揭示了医疗、能源及金融等行业在威胁覆盖上的显著缺口。其中,如CIC-IoMT和CIC-UNSW-NB15等近期数据集更贴近行业威胁,而即便更新的CIC-IoV-24也表现不佳。研究结果为选择符合行业实际需求的数据集提供了标准化、可解释的方法,显著提升AI驱动的IDS/IPS在真实环境中的有效性。框架已在真实案例中验证其高效性与实用性,证明其能有效指导数据选择并增强系统部署效果。

原文摘要 · Abstract (English)

The performance of Machine Learning (ML) and Deep Learning (DL)-based Intrusion Detection and Prevention Systems (IDS/IPS) is critically dependent on the relevance and quality of the datasets used for training and evaluation. However, current AI model evaluation practices for developing IDS/IPS focus predominantly on accuracy metrics, often overlooking whether datasets represent industry-specific threats. To address this gap, we introduce a novel multi-dimensional framework that integrates the MITRE ATT&CK knowledge base for threat intelligence and employs five complementary metrics that together provide a comprehensive assessment of dataset suitability. Methodologically, this framework combines threat intelligence, natural language processing, and quantitative analysis to assess the suitability of datasets for specific industry contexts. Applying this framework to nine publicly available IDS/IPS datasets reveals significant gaps in threat coverage, particularly in the healthcare, energy, and financial sectors. In particular, recent datasets (e.g., CIC-IoMT, CIC-UNSW-NB15) align better with sector-specific threats, whereas others, like CICIoV-24, underperform despite their recency. Our findings provide a standardized, interpretable approach for selecting datasets aligned with sector-specific operational requirements, ultimately enhancing the real-world effectiveness of AI-driven IDS/IPS deployments. The efficiency and practicality of the framework are validated through deployment in a real-world case study, underscoring its capacity to inform dataset selection and enhance the effectiveness of AI-driven IDS/IPS in operational environments.

入侵检测数据集评估MITRE ATT&CK行业安全

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。