通过梯度动态分析实现无监督异常检测的早停,防止模型过拟合。
GradStop: Exploring Training Dynamics in Unsupervised Outlier Detection through Gradient
- 基于梯度凝聚度设计采样算法,实时评估训练中的异常检测性能。
- 在47个真实数据集上提升4种深度方法性能,最优比现有最佳还高12.3%。
- 无需标签,适合对模型泛化性要求高的工业异常检测场景。
无监督异常检测(UOD)是数据挖掘与机器学习中的关键任务,旨在识别显著偏离多数样本的实例。由于缺乏标签,深度UOD方法常因模型直接优化目标与最终异常检测性能目标不一致而表现不佳。本文从训练动态角度出发,提出一种早停算法GradStop,以优化深度UOD模型训练,避免其过度拟合污染数据集。受UOD机制及内点优先现象启发——模型通常更快拟合内点而非异常点——我们设计了一种基于采样的无标签算法,实时估计模型性能。首先通过采样生成两组样本:一组含较多异常点,另一组含较多内点;再利用基于梯度凝聚度的指标探测当前训练动态,反映模型在异常检测任务上的实际表现。在4种深度UOD算法和47个真实数据集上的实验结果,以及理论证明均验证了该方法的有效性。使用GradStop增强的自编码器(AE)性能优于自身、其他SOTA UOD方法,甚至超过集成AE。本方法为缓解训练过程中的性能退化问题提供了鲁棒有效的解决方案,使深度UOD模型在异常检测任务中发挥更大潜力。
原文摘要 · Abstract (English)
Unsupervised Outlier Detection (UOD) is a critical task in data mining and machine learning, aiming to identify instances that significantly deviate from the majority. Without any label, deep UOD methods struggle with the misalignment between the model's direct optimization goal and the final performance goal of Outlier Detection (OD) task. Through the perspective of training dynamics, this paper proposes an early stopping algorithm to optimize the training of deep UOD models, ensuring they perform optimally in OD rather than overfitting the entire contaminated dataset. Inspired by UOD mechanism and inlier priority phenomenon, where intuitively models fit inliers more quickly than outliers, we propose GradStop, a sampling-based label-free algorithm to estimate model's real-time performance during training. First, a sampling method generates two sets: one likely containing more outliers and the other more inliers, then a metric based on gradient cohesion is applied to probe into current training dynamics, which reflects model's performance on OD task. Experimental results on 4 deep UOD algorithms and 47 real-world datasets and theoretical proofs demonstrate the effectiveness of our proposed early stopping algorithm in enhancing the performance of deep UOD models. Auto Encoder (AE) enhanced by GradStop achieves better performance than itself, other SOTA UOD methods, and even ensemble AEs. Our method provides a robust and effective solution to the problem of performance degradation during training, enabling deep UOD models to achieve better potential in anomaly detection tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。