arXiv:2503.01260cs.LG2025-03被引 1

提出新型时间序列异常检测评估方法,更贴近真实运维场景。

OIPR: Evaluation for Time-series Anomaly Detection Inspired by Operator Interest

  • 基于运维人员关注区域设计面积型评估指标
  • 在复杂场景下平衡点与事件评价的优缺点
  • 适合评估工业、物联网等真实系统中的异常检测器

随着时间序列异常检测(TAD)技术在互联网服务、工业系统和传感器领域的广泛应用,检测器的选型与优化严重依赖有效的性能评估。由于时间序列异常常表现为一段连续的异常点,仅关注单个点的评估指标已不充分。现有评估方法多采用点或事件为基础的指标,但点评估易高估对长异常检测能力强的模型,而事件评估则易受碎片化检测结果干扰。为此,本文提出一种基于运维人员关注兴趣的新型评估指标OIPR(Operator Interest-based Precision and Recall),采用面积型指标建模运维人员接收报警并处理异常的过程。同时构建特殊场景数据集用于对比不同评估方法特性。在该特殊数据集及五个真实世界数据集上的实验表明,OIPR在极端和复杂场景中表现优异,有效平衡了点与事件视角的局限性,具备更广泛的应用潜力。

原文摘要 · Abstract (English)

With the growing adoption of time-series anomaly detection (TAD) technology, numerous studies have employed deep learning-based detectors to analyze time-series data in the fields of Internet services, industrial systems, and sensors. The selection and optimization of anomaly detectors strongly rely on the availability of an effective evaluation for TAD performance. Since anomalies in time-series data often manifest as a sequence of points, conventional metrics that solely consider the detection of individual points are inadequate. Existing TAD evaluators typically employ point-based or event-based metrics to capture the temporal context. However, point-based evaluators tend to overestimate detectors that excel only in detecting long anomalies, while event-based evaluators are susceptible to being misled by fragmented detection results. To address these limitations, we propose OIPR (Operator Interest-based Precision and Recall metrics), a novel TAD evaluator with area-based metrics. It models the process of operators receiving detector alarms and handling anomalies, utilizing area under the operator interest curve to evaluate TAD performance. Furthermore, we build a special scenario dataset to compare the characteristics of different evaluators. Through experiments conducted on the special scenario dataset and five real-world datasets, we demonstrate the remarkable performance of OIPR in extreme and complex scenarios. It achieves a balance between point and event perspectives, overcoming their primary limitations and offering applicability to broader situations.

异常检测时间序列评估指标

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。