arXiv:2606.07789cs.LGstat.ML2026-06KDD

提出新评估框架,让概念漂移检测方法公平对比。

A Framework for Evaluating and Benchmarking Concept Drift Detection Methods

  • 用蒙特卡洛法在真实数据中注入可控漂移,保持数据复杂性。
  • 设计时序敏感指标,量化检测准确率与延迟,支持跨数据流比较。
  • 建议留一数据集调参,提升模型在不同场景下的鲁棒性。

数据流挖掘面临概念漂移的挑战,分布变化会降低模型性能。尽管漂移检测方法众多,但研究进展受限于不一致的评估方式:多数使用简化合成数据、采用不兼容指标、超参数选择缺乏透明度,导致公平比较困难。本文提出一个新基准框架,包含三方面贡献:(1) 通过蒙特卡洛试验将受控分布变化注入真实数据集,实现监督评估的同时保留真实数据复杂性;(2) 设计时序感知的评估协议,引入新指标(如F1检测得分、归一化检测时间),支持跨数据流可比性;(3) 提出留一数据集外的超参数优化协议,提升方法在异构流动态下的配置鲁棒性。我们在7个真实数据集上,对4种漂移类型(类别先验、标签交换、特征置换、特征过滤)的14种主流检测方法进行评估,涵盖突发和渐进式变化。实验揭示了现有方法的优劣,并为未来研究建立了基线性能标准。所有代码与实验公开可用。

原文摘要 · Abstract (English)

Data stream mining is fundamentally challenged by concept drift, where distributional changes can degrade model performance. Despite the proliferation of drift detection methods, progress in the field is hindered by inconsistent evaluation practices: studies rely on oversimplified synthetic data generators, adopt incompatible metrics, and lack transparency in hyperparameter selection, making fair comparisons difficult. We address this gap with a novel benchmarking framework comprising three contributions: (1) a drift simulation method that injects controlled distributional changes into real-world datasets via Monte Carlo trials, enabling supervised evaluation while preserving real-world data complexity; (2) an evaluation protocol for drift detection with timing-aware criteria, including the derivation of new metrics (e.g., F1 detection score, normalized detection time) that are comparable across streams; and (3) we advocate for a leave-one-dataset-out hyperparameter optimization protocol for drift detection methods that promotes configuration robustness across heterogeneous stream dynamics. We benchmark 14 widely used drift detection methods on 7 realworld datasets across 4 drift types (class prior, label swap, feature permutation, feature filtering), each under both abrupt and gradual transitions. Our experimental results provide insights into the strengths and weaknesses of current drift detection approaches while establishing baseline performance metrics for future research in this area. All code and experiments are publicly available.

概念漂移评估框架数据流

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。