通过评估数据噪声量，无需异常样本即可有效检测表格数据中的异常
Unsupervised Anomaly Detection for Tabular Data Using Noise Evaluation
- 用干净数据和加了多样噪声的数据训练神经网络，学习正常与异常的边界
- 在60多个基准数据集上平均达到92.27% AUC和1.68排名得分
- 理论可保证检测效果，实现简单，适合实际应用
无监督异常检测（UAD）在现代数据分析中至关重要，需要简单有效且有保障的算法。本文提出一种针对表格数据的新UAD方法，通过评估数据中的噪声程度实现。具体地，从干净（正常）数据集和通过添加高度多样噪声生成的噪声数据集中训练深度神经网络。当噪声数据多样性足够高时，硬异常样本会落入噪声区域，模型能可靠地学习正常与异常之间的决策边界。重要的是，我们提供了理论保证：即使训练阶段未使用真实异常数据，该方法仍可成功检测异常。在超过60个基准数据集上的大量实验表明，该方法优于12种现有基线，平均AUC达92.27%，排名得分为1.68。此外，相比当前最优方法，本方法更易实现。
原文摘要 · Abstract (English)
Unsupervised anomaly detection (UAD) plays an important role in modern data analytics and it is crucial to provide simple yet effective and guaranteed UAD algorithms for real applications. In this paper, we present a novel UAD method for tabular data by evaluating how much noise is in the data. Specifically, we propose to learn a deep neural network from the clean (normal) training dataset and a noisy dataset, where the latter is generated by adding highly diverse noises to the clean data. The neural network can learn a reliable decision boundary between normal data and anomalous data when the diversity of the generated noisy data is sufficiently high so that the hard abnormal samples lie in the noisy region. Importantly, we provide theoretical guarantees, proving that the proposed method can detect anomalous data successfully, although the method does not utilize any real anomalous data in the training stage. Extensive experiments through more than 60 benchmark datasets demonstrate the effectiveness of the proposed method in comparison to 12 baselines of UAD. Our method obtains a 92.27\% AUC score and a 1.68 ranking score on average. Moreover, compared to the state-of-the-art UAD methods, our method is easier to implement.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。