提出两种能自动忽略难分类样本的鲁棒损失函数,提升错误标签检测效果。
Hard Samples, Bad Labels: Robust Loss Functions That Know When to Back Off
- 通过降低或忽略难分类样本的权重,减少错误标签对模型的影响。
- 在多种人工污染数据集上,检测准确率(F1)显著优于现有方法。
- 适合需要清洗训练数据的从业者,尤其适用于标签错误分布不均场景。
错误标注的训练数据在基准和精心构建的数据集中普遍存在。这类标注错误明显损害监督学习模型的性能与泛化能力。现有标签错误检测框架通常依赖于训练良好的模型,但这些模型又需在污染数据上训练,导致泛化能力下降,进而影响检测效果——除非采用对标签错误具有鲁棒性的训练方案。本文评估了两种新型损失函数:Blurry Loss 和 Piecewise-zero Loss,它们通过降低或忽略难以分类的样本(这些样本更可能被误标)的贡献,提升对标签错误的鲁棒性。大量实验表明,在多种人工污染数据集上,所提损失函数在几乎所有情况下均优于当前最优的鲁棒损失函数,实现更高的错误检测F1分数。消融研究进一步验证了其在均匀与非均匀污染情形下,以及不同错误检测框架中的广泛适用性。使用这些鲁棒损失函数,机器学习从业者可更有效地识别、剔除或修正训练数据中的错误。
原文摘要 · Abstract (English)
Incorrectly labelled training data are frustratingly ubiquitous in both benchmark and specially curated datasets. Such mislabelling clearly adversely affects the performance and generalizability of models trained through supervised learning on the associated datasets. Frameworks for detecting label errors typically require well-trained / well-generalized models; however, at the same time most frameworks rely on training these models on corrupt data, which clearly has the effect of reducing model generalizability and subsequent effectiveness in error detection -- unless a training scheme robust to label errors is employed. We evaluate two novel loss functions, Blurry Loss and Piecewise-zero Loss, that enhance robustness to label errors by de-weighting or disregarding difficult-to-classify samples, which are likely to be erroneous. These loss functions leverage the idea that mislabelled examples are typically more difficult to classify and should contribute less to the learning signal. Comprehensive experiments on a variety of artificially corrupted datasets demonstrate that the proposed loss functions outperform state-of-the-art robust loss functions in nearly all cases, achieving superior F1 scores for error detection. Further analyses through ablation studies offer insights to confirm these loss functions' broad applicability to cases of both uniform and non-uniform corruption, and with different label error detection frameworks. By using these robust loss functions, machine learning practitioners can more effectively identify, prune, or correct errors in their training data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。