用图结构分层参考集,精准识别散点与簇状异常
Hierarchical Reference Sets for Robust Unsupervised Detection of Scattered and Clustered Outliers
- 构建局部与全局参考集,通过图结构多角度评估异常
- 在真实物联网数据上显著提升散点与簇状异常检测率
- 适合处理含干扰性簇状异常的无监督异常检测任务
多数现实世界物联网数据分析任务为无监督,极易受异常值影响。除由传感器故障等引起的孤立散点异常外,物联网系统还常出现簇状异常——多个设备因局部干扰、安全威胁或区域误报产生相似异常数据,形成微聚类。这类簇状异常因局部密度高,易被误判为正常行为,掩盖了散点与上下文异常的检测。为此,我们提出一种新型异常检测范式,利用图结构挖掘自然邻近关系,构建局部与全局参考集,实现多视角异常评估。该方法可有效识别散点异常,同时反映并隔离簇状异常群组。大量实验包括对比性能分析、消融研究、下游聚类任务验证及超参数敏感性测试均证明方法有效性。源代码见 https://github.com/gordonlok/DROD。
原文摘要 · Abstract (English)
Most real-world IoT data analysis tasks, such as clustering and anomaly event detection, are unsupervised and highly susceptible to the presence of outliers. In addition to sporadic scattered outliers caused by factors such as faulty sensor readings, IoT systems often exhibit clustered outliers. These occur when multiple devices or nodes produce similar anomalous measurements, for instance, owing to localized interference, emerging security threats, or regional false alarms, forming micro-clusters. These clustered outliers can be easily mistaken for normal behavior because of their relatively high local density, thereby obscuring the detection of both scattered and contextual anomalies. To address this, we propose a novel outlier detection paradigm that leverages the natural neighboring relationships using graph structures. This facilitates multi-perspective anomaly evaluation by incorporating reference sets at both local and global scales derived from the graph. Our approach enables the effective recognition of scattered outliers without interference from clustered anomalies, whereas the graph structure simultaneously helps reflect and isolate clustered outlier groups. Extensive experiments, including comparative performance analysis, ablation studies, validation on downstream clustering tasks, and evaluation of hyperparameter sensitivity, demonstrate the efficacy of the proposed method. The source code is available at https://github.com/gordonlok/DROD.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。