提出ODAR方法,让普通聚类算法自动识别异常值。
Detecting outliers by clustering algorithms
- 通过特征变换将异常值与正常数据分到不同簇
- 在10个数据集上使聚类准确率提升至少5%
- 无需额外步骤,通用性强,适合各类聚类算法
聚类与异常检测是数据挖掘中的重要任务。异常值常干扰聚类算法对对象相似性的判断,导致结果不可靠。目前仅有少数算法(如DBSCAN)具备异常检测能力,其他算法需额外处理异常值,过程繁琐。现有基于距离的异常检测策略会损害聚类性能。本文提出新型异常检测方法ODAR:通过特征变换将异常值与正常对象映射至两个分离簇,使任意聚类算法均可通过识别簇来检测异常。实验表明,ODAR对多样化数据集具有鲁棒性;在10个数据集中的7个上,结合ODAR的聚类算法性能达到最优,准确率提升至少5%。
原文摘要 · Abstract (English)
Clustering and outlier detection are two important tasks in data mining. Outliers frequently interfere with clustering algorithms to determine the similarity between objects, resulting in unreliable clustering results. Currently, only a few clustering algorithms (e.g., DBSCAN) have the ability to detect outliers to eliminate interference. For other clustering algorithms, it is tedious to introduce another outlier detection task to eliminate outliers before each clustering process. Obviously, how to equip more clustering algorithms with outlier detection ability is very meaningful. Although a common strategy allows clustering algorithms to detect outliers based on the distance between objects and clusters, it is contradictory to improving the performance of clustering algorithms on the datasets with outliers. In this paper, we propose a novel outlier detection approach, called ODAR, for clustering. ODAR maps outliers and normal objects into two separated clusters by feature transformation. As a result, any clustering algorithm can detect outliers by identifying clusters. Experiments show that ODAR is robust to diverse datasets. Compared with baseline methods, the clustering algorithms achieve the best on 7 out of 10 datasets with the help of ODAR, with at least 5% improvement in accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。