arXiv:2501.01130cs.LG2025-01被引 4

提出统一理论框架,让对比学习更抗标签噪声。

An Inclusive Theoretical Framework of Robust Supervised Contrastive Loss against Label Noise

  • 建立首个通用鲁棒性判据,可验证对比损失的抗噪能力。
  • 发现经典InfoNCE不鲁棒,提出改进版对称InfoNCE(SymNCE)。
  • 适用于数据清洗、弱监督学习等标签噪声场景。

从带噪声标签中学习是机器学习中的关键挑战,影响众多现实应用。尽管监督对比学习近年来成为应对标签噪声的强大工具,但多数现有方法仍依赖启发式设计,缺乏系统性的理论基础来构建鲁棒的监督对比损失。为此,本文首次提出了基于成对对比范式的鲁棒损失统一理论框架。我们首次推导出任意对比损失的通用鲁棒条件,可作为检验监督对比损失抗标签噪声理论鲁棒性的标准。理论表明,流行的InfoNCE损失实际上并不鲁棒,据此我们进一步提出改进版本——对称InfoNCE(SymNCE)。此外,该理论具有包容性,能解释先前的鲁棒技术,如最近邻样本选择和鲁棒对比损失。在基准数据集上的验证实验表明,SymNCE在标签噪声下表现显著优于现有方法。

原文摘要 · Abstract (English)

Learning from noisy labels is a critical challenge in machine learning, with vast implications for numerous real-world scenarios. While supervised contrastive learning has recently emerged as a powerful tool for navigating label noise, many existing solutions remain heuristic, often devoid of a systematic theoretical foundation for crafting robust supervised contrastive losses. To address the gap, in this paper, we propose a unified theoretical framework for robust losses under the pairwise contrastive paradigm. In particular, we for the first time derive a general robust condition for arbitrary contrastive losses, which serves as a criterion to verify the theoretical robustness of a supervised contrastive loss against label noise. The theory indicates that the popular InfoNCE loss is in fact non-robust, and accordingly inspires us to develop a robust version of InfoNCE, termed Symmetric InfoNCE (SymNCE). Moreover, we highlight that our theory is an inclusive framework that provides explanations to prior robust techniques such as nearest-neighbor (NN) sample selection and robust contrastive loss. Validation experiments on benchmark datasets demonstrate the superiority of SymNCE against label noise.

对比学习标签噪声理论分析鲁棒训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。