提出SUDS策略，用无监督方法选样本重训模型，应对数据漂移。
SUDS: A Strategy for Unsupervised Drift Sampling
- 基于已有漂移检测算法，选择同质样本用于重训。
- 实验显示显著提升动态环境下模型性能。
- 适合需要高效标注资源的实时学习场景。
监督机器学习常面临概念漂移问题，即数据分布随时间变化导致模型性能下降。现有漂移检测方法多关注识别漂移，却忽视漂移后获取标注数据以重训模型的难题。本文提出漂移采样策略（SUDS），利用现有漂移检测算法筛选同质样本用于模型重训，增强模型对演化数据的适应能力。SUDS可无缝集成至当前漂移检测技术中。同时引入调和标注数据准确率度量（HADAM），评估分类器性能与所需标注数据量之间的平衡，体现标注获取难度。贡献有二：一是将漂移检测与策略性采样结合优化重训流程；二是提出兼顾性能与标注成本的评估指标。实验证明SUDS能有效优化动态环境中标注数据使用，显著提升真实场景下机器学习应用表现。代码开源，详见 https://github.com/cfellicious/SUDS/
原文摘要 · Abstract (English)
Supervised machine learning often encounters concept drift, where the data distribution changes over time, degrading model performance. Existing drift detection methods focus on identifying these shifts but often overlook the challenge of acquiring labeled data for model retraining after a shift occurs. We present the Strategy for Drift Sampling (SUDS), a novel method that selects homogeneous samples for retraining using existing drift detection algorithms, thereby enhancing model adaptability to evolving data. SUDS seamlessly integrates with current drift detection techniques. We also introduce the Harmonized Annotated Data Accuracy Metric (HADAM), a metric that evaluates classifier performance in relation to the quantity of annotated data required to achieve the stated performance, thereby taking into account the difficulty of acquiring labeled data. Our contributions are twofold: SUDS combines drift detection with strategic sampling to improve the retraining process, and HADAM provides a metric that balances classifier performance with the amount of labeled data, ensuring efficient resource utilization. Empirical results demonstrate the efficacy of SUDS in optimizing labeled data use in dynamic environments, significantly improving the performance of machine learning applications in real-world scenarios. Our code is open source and available at https://github.com/cfellicious/SUDS/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。