结合聚类与专家视觉判断,提升跨领域异常检测准确性。
A method for outlier detection based on cluster analysis and visual expert criteria
- 基于聚类后的人类专家视觉标准识别异常点。
- 在两个领域测试中错误率低于2%,可靠性超99%。
- 适合医疗信号分析等需人工经验辅助的场景。
异常检测广泛存在于多个领域,常由欺诈行为、机械故障或人为误差导致。许多数据挖掘应用将其作为预处理步骤,以构建更代表性的模型。本文提出一种基于聚类分析的异常检测方法,旨在克服现有技术忽视领域对象固有分散性的缺陷。该方法采用四项由各领域专家根据聚类后视觉判断设计的标准,相比纯数值分析更具可解释性。在两个不同领域的数据上进行了验证:平衡学(stabilometry)和脑电图(EEG),均为时间序列数据。结果表明,该方法在异常检测效率与运行时间方面表现良好,回归分析证实其有效性:错误率低于2%,可靠性超过99%。
原文摘要 · Abstract (English)
Outlier detection is an important problem occurring in a wide range of areas. Outliers are the outcome of fraudulent behaviour, mechanical faults, human error, or simply natural deviations. Many data mining applications perform outlier detection, often as a preliminary step in order to filter out outliers and build more representative models. In this paper, we propose an outlier detection method based on a clustering process. The aim behind the proposal outlined in this paper is to overcome the specificity of many existing outlier detection techniques that fail to take into account the inherent dispersion of domain objects. The outlier detection method is based on four criteria designed to represent how human beings (experts in each domain) visually identify outliers within a set of objects after analysing the clusters. This has an advantage over other clustering-based outlier detection techniques that are founded on a purely numerical analysis of clusters. Our proposal has been evaluated, with satisfactory results, on data (particularly time series) from two different domains: stabilometry, a branch of medicine studying balance-related functions in human beings and electroencephalography (EEG), a neurological exploration used to diagnose nervous system disorders. To validate the proposed method, we studied method outlier detection and efficiency in terms of runtime. The results of regression analyses confirm that our proposal is useful for detecting outlier data in different domains, with a false positive rate of less than 2% and a reliability greater than 99%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。