用改进的逻辑损失提升视觉表征学习，无需全局视图。
SigCLR: Sigmoid Contrastive Learning of Visual Representations
- 采用仅处理样本对的逻辑损失，替代SimCLR的交叉熵损失。
- 在CIFAR-10、CIFAR-100和Tiny-IN上性能媲美现有自监督方法。
- 需固定温度参数，适合追求高效无全局视图的表征学习场景。
我们提出SigCLR：一种基于逻辑损失的视觉表征对比学习方法。与SimCLR使用的交叉熵损失不同，该方法仅作用于样本对，无需全局视图。实验表明，在CIFAR-10、CIFAR-100和Tiny-IN数据集上，该方法性能可与现有自监督学习目标相媲美。研究验证了可学习偏置的重要性，但要取得优异表现，仍需像SimCLR一样采用固定温度参数。总体而言，SigCLR是一种极具潜力的SimCLR替代方案，尤其适用于广泛部署且已证明在多个领域成功的自监督学习任务。
原文摘要 · Abstract (English)
We propose SigCLR: Sigmoid Contrastive Learning of Visual Representations. SigCLR utilizes the logistic loss that only operates on pairs and does not require a global view as in the cross-entropy loss used in SimCLR. We show that logistic loss shows competitive performance on CIFAR-10, CIFAR-100, and Tiny-IN compared to other established SSL objectives. Our findings verify the importance of learnable bias as in the case of SigLUP, however, it requires a fixed temperature as in the SimCLR to excel. Overall, SigCLR is a promising replacement for the SimCLR which is ubiquitous and has shown tremendous success in various domains.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。