通过随机旋转数据提升异常检测精度,解决传统方法的伪簇问题。
Detecting Anomalies Using Rotated Isolation Forest
- 在iForest前加入随机旋转数据,增强样本多样性
- 在合成与真实数据集上均显著优于iForest和EIF
- 适合需要高精度无监督异常检测的场景
Isolation Forest(iForest)自2012年提出以来成为主流无监督异常检测方法。然而,2021年Hariri等人发现iForest存在轴对齐伪簇问题,会导致异常评分偏差。为此他们提出Extended Isolation Forest(EIF),有效消除此类伪簇,提升评分一致性与性能。本文揭示EIF仍存在正常簇间的伪间簇问题。为此提出旋转隔离森林(RIF),通过在iForest构建前对数据集施加随机旋转矩阵及QR分解,增加数据变异性,有效消除iForest的轴对齐伪簇与EIF的伪间簇。实验表明,RIF在合成数据与真实数据集上均显著优于iForest和EIF。
原文摘要 · Abstract (English)
The Isolation Forest (iForest), proposed by Liu, Ting, and Zhou at TKDE 2012, has become a prominent tool for unsupervised anomaly detection. However, recent research by Hariri, Kind, and Brunner, published in TKDE 2021, has revealed issues with iForest. They identified the presence of axis-aligned ghost clusters that can be misidentified as normal clusters, leading to biased anomaly scores and inaccurate predictions. In response, they developed the Extended Isolation Forest (EIF), which effectively solves these issues by eliminating the ghost clusters introduced by iForest. This enhancement results in improved consistency of anomaly scores and superior performance. We reveal a previously overlooked problem in the Extended Isolation Forest (EIF), showing that it is vulnerable to ghost inter-clusters between normal clusters of data points. In this paper, we introduce the Rotated Isolation Forest (RIF) algorithm which effectively addresses both the axis-aligned ghost clusters observed in iForest and the ghost inter-clusters seen in EIF. RIF accomplishes this by randomly rotating the dataset (using random rotation matrices and QR decomposition) before feeding it into the iForest construction, thereby increasing dataset variation and eliminating ghost clusters. Our experiments conclusively demonstrate that the RIF algorithm outperforms iForest and EIF, as evidenced by the results obtained from both synthetic datasets and real-world datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。