统一评估框架让入侵检测系统对比更公平可靠
PIDSMaker: Building and Evaluating Provenance-based Intrusion Detection Systems
- 构建标准化流程，整合8个先进检测系统
- 支持快速原型搭建与可复现实验
- 适合研究者快速验证新方法或对比模型
近期基于溯源的入侵检测系统（PIDS）通过机器学习分析系统溯源图，在检测高级持续性威胁（APTs）方面展现出强大潜力。然而，现有研究在评估和比较时存在预处理不一致、数据集划分非标准、标注与指标不兼容等问题，严重影响可复现性、阻碍公平比较，并增加研究者重实现成本。本文提出PIDSMaker，一个开源框架，用于在统一协议下开发与评估PIDS。该框架将8个前沿系统整合为模块化、可扩展架构，提供标准化预处理和真实标签，支持一致实验与直接比较。通过基于YAML的配置接口，无需代码修改即可组合各系统组件，实现快速原型设计。框架还包含消融实验、超参数调优、多轮运行稳定性测量及可视化工具，填补了先前研究的方法论空白。我们通过具体用例验证了其有效性，并发布预处理数据集与标签，推动PIDS领域的共享评估。
原文摘要 · Abstract (English)
Recent provenance-based intrusion detection systems (PIDSs) have demonstrated strong potential for detecting advanced persistent threats (APTs) by applying machine learning to system provenance graphs. However, evaluating and comparing PIDSs remains difficult: prior work uses inconsistent preprocessing pipelines, non-standard dataset splits, and incompatible ground-truth labeling and metrics. These discrepancies undermine reproducibility, impede fair comparison, and impose substantial re-implementation overhead on researchers. We present PIDSMaker, an open-source framework for developing and evaluating PIDSs under consistent protocols. PIDSMaker consolidates eight state-of-the-art systems into a modular, extensible architecture with standardized preprocessing and ground-truth labels, enabling consistent experiments and apples-to-apples comparisons. A YAML-based configuration interface supports rapid prototyping by composing components across systems without code changes. PIDSMaker also includes utilities for ablation studies, hyperparameter tuning, multi-run instability measurement, and visualization, addressing methodological gaps identified in prior work. We demonstrate PIDSMaker through concrete use cases and release it with preprocessed datasets and labels to support shared evaluation for the PIDS community.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。