arXiv:2502.07551cs.LG2025-02ICLR被引 47

无需验证集,通过预测波动检测噪声并提前停止训练

Early Stopping Against Label Noise Without Validation Data

  • 监控训练集上模型预测的波动变化来决定停止时机
  • 在多种噪声环境下显著提升模型性能,优于传统方法
  • 适合无验证数据或标签噪声严重的实际场景

深度学习中的早停方法面临训练与验证数据分配的权衡问题,尤其在标签噪声存在时更为突出。保留更多数据用于验证会限制模型性能,而验证数据不足则可能导致模型选择不佳。本文提出一种名为 Label Wave 的新早停方法,可在无验证数据的情况下应对标签噪声。该方法通过追踪训练过程中模型在训练集上的预测变化,旨在模型过度拟合错误标签前停止训练。实验表明,预测波动最小的时刻通常出现在模型过度拟合错误标签之前。大量实验证明,Label Wave 在多种设置下均有效,且能显著提升现有噪声标签学习方法的性能。

原文摘要 · Abstract (English)

Early stopping methods in deep learning face the challenge of balancing the volume of training and validation data, especially in the presence of label noise. Concretely, sparing more data for validation from training data would limit the performance of the learned model, yet insufficient validation data could result in a sub-optimal selection of the desired model. In this paper, we propose a novel early stopping method called Label Wave, which does not require validation data for selecting the desired model in the presence of label noise. It works by tracking the changes in the model's predictions on the training set during the training process, aiming to halt training before the model unduly fits mislabeled data. This method is empirically supported by our observation that minimum fluctuations in predictions typically occur at the training epoch before the model excessively fits mislabeled data. Through extensive experiments, we show both the effectiveness of the Label Wave method across various settings and its capability to enhance the performance of existing methods for learning with noisy labels.

早停标签噪声无验证集

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。