评估漂移检测质量时，分类准确率可能不可靠，需用专门指标。
How well does Classification Accuracy capture Concept Drift Detection Quality? An overview of Concept Drift Detection evaluation
- 用8种专业指标对比分类器在7个生成工具上的表现
- 发现准确率与漂移检测效果相关性不强
- 适合研究数据流和模型评估的学者参考
数据流是当前最常分析的数据结构之一，概念漂移是处理系统面临的主要挑战。尽管已有众多方法应对漂移导致的性能下降，但学术界尚未建立统一的概念漂移检测评估框架。现有研究多依赖分类性能指标，但这些指标受多种因素影响，未必能真实反映漂移检测质量。本文深入分析了漂移检测质量指标与分类性能在合成非平稳数据流中的关系。研究考察了8种漂移检测质量指标，在7种合成数据流生成工具下的表现，并考虑漂移动态作为变量。目标是识别最有效的漂移检测评估指标集，并深化对评估方法的理解。
原文摘要 · Abstract (English)
Data streams are nowadays among the most frequently analyzed data structures, with the concept drift posing a major challenge encountered by processing systems. Despite the proposition of numerous solutions to counteract the accuracy degeneration due to concept drift, the scientific community has not yet established a unified framework for evaluating the concept drift detection task. Existing research often relies on classification quality metrics, but these can be affected by multiple factors and may not reliably reflect drift detection quality. In this work, we present an in-depth overview of the relationship between metrics for quantifying drift detection quality and classification performance in synthetic nonstationary data streams. The proposed research studies eight drift detection quality metrics in relation to the classifier's performance across seven synthetic data stream generation tools, additionally considering drift dynamics as a factor. The studies aim to identify the most informative set of drift detection quality metrics and provide a deep understanding of the method's evaluation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。