用数学函数检测数据流异常,无需调参且省内存。
Leveraging the Christoffel Function for Outlier Detection in Data Streams
- 基于正交多项式理论的Christoffel函数,构建无参数异常检测方法。
- 新方法在真实工业数据上比调参方法快30%以上,内存占用低。
- 适合实时数据流处理,尤其对参数难调的场景友好。
异常检测在数据挖掘中至关重要,尤其在数据采集日益普及的背景下。识别数据流中的异常对于保障数据质量与故障检测极为关键。然而,数据流的非平稳分布和持续增长的数据量带来了挑战。尽管已有多种方法,但多数缺乏直观的参数设定。本文提出两种新方法:DyCF 和 DyCG。DyCF 利用逼近论与正交多项式中的 Christoffel 函数;DyCG 则利用该函数的增长特性,无需调参。二者均建立在明确的代数框架之上,满足数据流处理需求,尤其关注低维数据,且不增加内存开销。通过合成与真实工业数据流的对比实验表明,DyCF 在执行时间与内存使用上优于需调参的方法;DyCG 虽性能略低,但完全无需调参,优势显著。
原文摘要 · Abstract (English)
Outlier detection holds significant importance in the realm of data mining, particularly with the growing pervasiveness of data acquisition methods. The ability to identify outliers in data streams is essential for maintaining data quality and detecting faults. However, dealing with data streams presents challenges due to the non-stationary nature of distributions and the ever-increasing data volume. While numerous methods have been proposed to tackle this challenge, a common drawback is the lack of straightforward parameterization in many of them. This article introduces two novel methods: DyCF and DyCG. DyCF leverages the Christoffel function from the theory of approximation and orthogonal polynomials. Conversely, DyCG capitalizes on the growth properties of the Christoffel function, eliminating the need for tuning parameters. Both approaches are firmly rooted in a well-defined algebraic framework, meeting crucial demands for data stream processing, with a specific focus on addressing low-dimensional aspects and maintaining data history without memory cost. A comprehensive comparison between DyCF, DyCG, and state-of-the-art methods is presented, using both synthetic and real industrial data streams. The results show that DyCF outperforms fine-tuning methods, offering superior performance in terms of execution time and memory usage. DyCG performs less well, but has the considerable advantage of requiring no tuning at all.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。