通过流形视角提升高维数据异常检测效果，融合方法显著提高召回率。
Finding Pegasus: Enhancing Unsupervised Anomaly Detection in High-Dimensional Data using a Manifold-Based Approach
- 基于流形结构区分异常在流形上或流形外，构建新分类框架
- 融合多种异常检测方法，在MNIST上召回率提升16%
- 适合高维数据场景，尤其适用于对召回率要求高的真实应用
无监督机器学习方法适用于大规模异常检测，但在现代高维数据的表示下表现受限，因此常先进行降维（DR）。本文从降维后形成的流形角度分析无监督异常检测（AD），提出理想化示例“Finding Pegasus”及一种新形式化框架，将AD方法及其结果划分为‘在流形上’和‘在流形外’两类，并阐明其差异。基于此洞察，我们提出一种融合多方法的异常检测策略，在高维降维场景下显著提升召回率且不牺牲精度。在MNIST数据集上的测试表明，该方法相比仅使用最优单个方法（孤立森林）的组合，召回率最高提升16%，展现出在真实数据中应用的巨大潜力。
原文摘要 · Abstract (English)
Unsupervised machine learning methods are well suited to searching for anomalies at scale but can struggle with the high-dimensional representation of many modern datasets, hence dimensionality reduction (DR) is often performed first. In this paper we analyse unsupervised anomaly detection (AD) from the perspective of the manifold created in DR. We present an idealised illustration, "Finding Pegasus", and a novel formal framework with which we categorise AD methods and their results into "on manifold" and "off manifold". We define these terms and show how they differ. We then use this insight to develop an approach of combining AD methods which significantly boosts AD recall without sacrificing precision in situations employing high DR. When tested on MNIST data, our approach of combining AD methods improves recall by as much as 16 percent compared with simply combining with the best standalone AD method (Isolation Forest), a result which shows great promise for its application to real-world data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。