用自监督预训练提升模型抗噪声标签能力，无需干净数据集。
Pre-train to Gain: Robust Learning Without Clean Labels
- 先无标签自监督预训练特征提取器，再在带噪数据上监督训练。
- 在合成与真实噪声下，准确率均显著提升，噪声越高优势越明显。
- 适合标签质量差但想用预训练模型的场景，尤其高噪声环境。
使用带噪声标签训练深度网络会导致过拟合，降低泛化性能。现有方法通常依赖干净数据子集。本文通过自监督学习（SSL）在无标签情况下预训练特征提取器，随后在带噪数据上进行标准监督训练，实现无需干净数据子集的鲁棒学习。在CIFAR-10和CIFAR-100上，采用SimCLR和Barlow Twins作为SSL方法，在合成与真实噪声下验证，所有噪声率下分类准确率均持续提升，下游标签错误检测能力（F1与平衡准确率）也增强。噪声率越高，性能差距越大，表明更强的鲁棒性。值得注意的是，在低噪声水平下表现接近ImageNet预训练模型，而在高噪声条件下显著超越它们。
原文摘要 · Abstract (English)
Training deep networks with noisy labels leads to poor generalization and degraded accuracy due to overfitting to label noise. Existing approaches for learning with noisy labels often rely on the availability of a clean subset of data. By pre-training a feature extractor backbone without labels using self-supervised learning (SSL), followed by standard supervised training on the noisy dataset, we can train a more noise robust model without requiring a subset with clean labels. We evaluate the use of SimCLR and Barlow~Twins as SSL methods on CIFAR-10 and CIFAR-100 under synthetic and real world noise. Across all noise rates, self-supervised pre-training consistently improves classification accuracy and enhances downstream label-error detection (F1 and Balanced Accuracy). The performance gap widens as the noise rate increases, demonstrating improved robustness. Notably, our approach achieves comparable results to ImageNet pre-trained models at low noise levels, while substantially outperforming them under high noise conditions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。