提出新型图结构算法,有效检测高维复杂形状数据中的异常点。
Outlier Detection with Cluster Catch Digraphs
- 基于聚类捕获有向图构建新算法,结合最近邻距离与均匀性判断。
- 在高维和非均匀聚类下表现稳定,真实数据验证了识别精度提升。
- 适合处理复杂分布数据,尤其适用于金融、医疗等高维异常检测场景。
本文提出一类基于聚类捕获有向图(CCDs)的新异常检测算法,专门应对高维数据和任意聚类形状带来的挑战。我们设计了基于均匀性的互捕获图算法(U-MCCD)、结合均匀性与邻居信息的算法(UN-MCCD),以及其形状自适应变体(SU-MCCD 和 SUN-MCCD),可有效识别具有任意聚类形状的高维数据中的异常点。通过大规模蒙特卡洛模拟评估性能,结果表明这些算法在不同污染水平下均具鲁棒性和有效性。实际数据集应用进一步验证其适用性:U-MCCD 在保持高真阴性率的同时高效识别异常点;SU-MCCD 显著提升对非均匀聚类的处理能力;而 UN-MCCD 与 SUN-MCCD 利用最近邻距离(NND)在高维空间中实现更优聚类与异常检测。实验表明,该系列算法在准确性与适应性上均有显著提升,为实际应用提供了有力工具。
原文摘要 · Abstract (English)
This paper introduces a novel family of outlier detection algorithms based on Cluster Catch Digraphs (CCDs), specifically tailored to address the challenges of high dimensionality and varying cluster shapes, which deteriorate the performance of most traditional outlier detection methods. We propose the Uniformity-Based CCD with Mutual Catch Graph (U-MCCD), the Uniformity- and Neighbor-Based CCD with Mutual Catch Graph (UN-MCCD), and their shape-adaptive variants (SU-MCCD and SUN-MCCD), which are designed to detect outliers in data sets with arbitrary cluster shapes and high dimensions. We present the advantages and shortcomings of these algorithms and provide the motivation or need to define each particular algorithm. Through comprehensive Monte Carlo simulations, we assess their performance and demonstrate the robustness and effectiveness of our algorithms across various settings and contamination levels. We also illustrate the use of our algorithms on various real-life data sets. The U-MCCD algorithm efficiently identifies outliers while maintaining high true negative rates, and the SU-MCCD algorithm shows substantial improvement in handling non-uniform clusters. Additionally, the UN-MCCD and SUN-MCCD algorithms address the limitations of existing methods in high-dimensional spaces by utilizing Nearest Neighbor Distances (NND) for clustering and outlier detection. Our results indicate that these novel algorithms offer substantial advancements in the accuracy and adaptability of outlier detection, providing a valuable tool for various real-world applications. Keyword: Outlier detection, Graph-based clustering, Cluster catch digraphs, $k$-nearest-neighborhood, Mutual catch graphs, Nearest neighbor distance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。