arXiv:2506.09810cs.LGcs.IT2025-06被引 2

提出ProjNCE统一监督与自监督对比学习,提升模型表现。

Generalizing Supervised Contrastive learning: A Projection Perspective

  • 引入投影函数和负样本调整项,统一监督与自监督对比目标。
  • 在图像与音频数据集上,优于SupCon和交叉熵训练。
  • 适合需要改进对比学习的科研与工程人员参考。

自监督对比学习(SSCL)已成为表示学习的重要范式,已有研究从互信息和几何视角展开。然而,监督对比学习(SupCon)在此背景下关注较少:例如,尽管SSCL中使用的InfoNCE是互信息(MI)的下界,但SupCon与MI的关系尚不明确。为此,本文提出ProjNCE,作为InfoNCE的泛化形式,通过引入投影函数和负样本调整项,统一了监督与自监督对比目标。我们证明了ProjNCE构成有效的互信息下界,并在类别嵌入的投影策略选择上提供更大灵活性。基于此,我们进一步探索了在SupCon中基于中心点的类别嵌入,测试多种投影方法。大量实验表明,ProjNCE在图像与音频数据集上始终优于SupCon和标准交叉熵训练。本工作从信息论与投影视角双重优化了SupCon,为以SupCon为基础的对比学习任务提供了广泛适用的改进方案。

原文摘要 · Abstract (English)

Self-supervised contrastive learning (SSCL) has emerged as a powerful paradigm for representation learning and has been studied from multiple perspectives, including mutual information and geometric viewpoints. However, supervised contrastive (SupCon) approaches have received comparatively little attention in this context: for instance, while InfoNCE used in SSCL is known to form a lower bound on mutual information (MI), the relationship between SupCon and MI remains unexplored. To address this gap, we introduce ProjNCE, a generalization of the InfoNCE loss that unifies supervised and self-supervised contrastive objectives by incorporating projection functions and an adjustment term for negative pairs. We prove that ProjNCE constitutes a valid MI bound and affords greater flexibility in selecting projection strategies for class embeddings. Building on this flexibility, we further explore the centroid-based class embeddings in SupCon by exploring a variety of projection methods. Extensive experiments on image and audio datasets demonstrate that ProjNCE consistently outperforms both SupCon and standard cross-entropy training. Our work thus refines SupCon along two complementary perspectives--information-theoretic and projection viewpoints--and offers broadly applicable improvements whenever SupCon serves as the foundational contrastive objective.

对比学习信息论投影机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。