提出CLOP损失函数，防止对比学习中嵌入维度坍缩。
Semi-Supervised Contrastive Learning with Orthonormal Prototypes
- 通过正交原型约束，构建类嵌入的正交线性子空间。
- 在图像分类和目标检测任务上提升性能，且对学习率更稳定。
- 适合关注半监督学习泛化与稳定性的研究者。
对比学习在深度学习中表现出强大能力，可通过不同分布样本的对比来学习有效表征。然而，在半监督和自监督设置中，嵌入向量趋向于低维空间（维度坍缩）仍是重大挑战。本文首次识别出一个关键学习率阈值：超过该阈值时，标准对比损失会收敛至坍缩解。基于此，我们提出CLOP——一种新型半监督损失函数，通过促进类别嵌入之间形成正交线性子空间，有效防止维度坍缩。在真实与合成数据集上的大量实验表明，CLOP在图像分类与目标检测任务中均提升性能，并在不同学习率和批量大小下表现出更强稳定性。
原文摘要 · Abstract (English)
Contrastive learning has emerged as a powerful method in deep learning, excelling at learning effective representations through contrasting samples from different distributions. However, dimensional collapse, where embeddings converge into a lower-dimensional space, poses a significant challenge, especially in semi-supervised and self-supervised setups. In this paper, we first identify a critical learning-rate threshold, beyond which standard contrastive losses converge to collapsed solutions. Building on these insights, we propose CLOP, a novel semi-supervised loss function designed to prevent dimensional collapse by promoting the formation of orthogonal linear subspaces among class embeddings. Through extensive experiments on real and synthetic datasets, we demonstrate that CLOP improves performance in image classification and object detection tasks while also exhibiting greater stability across different learning rates and batch sizes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。