提出理论框架防止监督对比学习中的类别坍缩
A Theoretical Framework for Preventing Class Collapse in Supervised Contrastive Learning
- 构建单纯形嵌入模型分析嵌入结构
- 发现超参数影响嵌入判别性，可避免类别坍缩
- 适用于需要稳定特征表示的研究者
监督对比学习（SupCL）是表征学习中的重要方法，结合了有监督与自监督损失。但两者平衡困难，失衡会导致类别坍缩，即同类别嵌入缺乏区分度。本文提出基于理论的预防策略，引入单纯形到单纯形嵌入模型（SSEM），建模最小化监督对比损失的所有嵌入结构。通过SSEM分析超参数对嵌入的影响，给出实际超参数选择建议以降低类别坍缩风险。理论结果在合成与真实数据集上均得到实证支持。
原文摘要 · Abstract (English)
Supervised contrastive learning (SupCL) has emerged as a prominent approach in representation learning, leveraging both supervised and self-supervised losses. However, achieving an optimal balance between these losses is challenging; failing to do so can lead to class collapse, reducing discrimination among individual embeddings in the same class. In this paper, we present theoretically grounded guidelines for SupCL to prevent class collapse in learned representations. Specifically, we introduce the Simplex-to-Simplex Embedding Model (SSEM), a theoretical framework that models various embedding structures, including all embeddings that minimize the supervised contrastive loss. Through SSEM, we analyze how hyperparameters affect learned representations, offering practical guidelines for hyperparameter selection to mitigate the risk of class collapse. Our theoretical findings are supported by empirical results across synthetic and real-world datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。