动态更新的核方法,实时检测数据流异常且速度快一个数量级。
IDK-S: Incremental Distributional Kernel for Streaming Anomaly Detection
- 基于核均值嵌入框架,动态构建分布表示,适应数据流演化。
- 相比全量重训练,计算开销大幅降低,检测精度与重训练模型相当。
- 适合需要高实时性与高准确率的在线异常检测场景。
数据流上的异常检测面临重大挑战:需在分布不断变化的情况下保持高检测精度,同时确保实时性。本文提出 $\ ext{IDK}\ ext{S}$,一种新型增量式分布核方法,用于流式异常检测。该方法在核均值嵌入框架下构建动态表示,优势源于两大创新:一是继承了隔离分布核(Isolation Distributional Kernel)的优势,其基于数据依赖核,在离线场景中显著优于 Isolation Forest 和 Local Outlier Factor;二是采用轻量级增量更新机制,相比全量重训练的基线策略,显著降低计算开销,且不牺牲检测精度——统计上等价于全量重训练模型。在十三个基准数据集上的大量实验表明,$\ ext{IDK}\ ext{S}$ 在检测精度上表现更优,运行速度比现有最先进方法快多个数量级。
原文摘要 · Abstract (English)
Anomaly detection on data streams presents significant challenges, requiring methods to maintain high detection accuracy among evolving distributions while ensuring real-time efficiency. Here we introduce $\mathcal{IDK}$-$\mathcal{S}$, a novel $\mathbf{I}$ncremental $\mathbf{D}$istributional $\mathbf{K}$ernel for $\mathbf{S}$treaming anomaly detection that effectively addresses these challenges by creating a new dynamic representation in the kernel mean embedding framework. The superiority of $\mathcal{IDK}$-$\mathcal{S}$ is attributed to two key innovations. First, it inherits the strengths of the Isolation Distributional Kernel, an offline detector that has demonstrated significant performance advantages over foundational methods like Isolation Forest and Local Outlier Factor due to the use of a data-dependent kernel. Second, it adopts a lightweight incremental update mechanism that significantly reduces computational overhead compared to the naive baseline strategy of performing a full model retraining. This is achieved without compromising detection accuracy, a claim supported by its statistical equivalence to the full retrained model. Our extensive experiments on thirteen benchmarks demonstrate that $\mathcal{IDK}$-$\mathcal{S}$ achieves superior detection accuracy while operating substantially faster, in many cases by an order of magnitude, than existing state-of-the-art methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。