用语义正例提升自监督学习泛化能力,效果优于传统增强正例。
The Impact of Semantic Pairs on Self-Supervised Representation Learning
- 构建语义正例数据集,隔离对比增强正例的影响。
- 在迁移学习和目标检测中,语义正例提升泛化性能,尤其对SimCLR最显著。
- 语义正例能诱导超越常规变换的鲁棒性,适合改进自监督模型设计。
实例判别通过将同一图像的不同增强视图视为正例来学习视觉表征。然而,这些同图正例可能保留背景、纹理、光照等无关相关性。语义正例(即同类别不同实例)可通过呈现跨多样化上下文的物体来减少此类相关性。以往研究常将语义正例与增强正例或错误邻居(错误匹配的语义对)混合使用,难以区分其独立影响。本文提出一个受控的实证研究,评估语义正例在自监督表示学习中的作用。从ImageNet-1K中构建两个匹配子集:一个基于增强正例的基线,另一个为人工标注的语义正例数据集,保持相同类别组成与训练对数量。在相同训练条件下,对比代表性对比与非对比型自监督方法。在迁移学习与目标检测评估中,语义正例预训练始终优于增强正例。额外消融实验表明,语义正例可引发超越标准变换管道的不变性。其中,对比学习对语义正例受益最大,尤其SimCLR相对提升最为明显。结果澄清了语义正例在自监督学习中的作用,为有效利用语义信息提供指导。
原文摘要 · Abstract (English)
Instance discrimination learns visual representations by treating different augmented views of the same image as positive pairs. While this encourages invariance to handcrafted transformations, same-image positives can preserve nuisance correlations such as background, texture, illumination, and object-specific details. Semantic positive pairs, i.e., different same-class instances, may reduce these correlations by presenting objects across diverse contexts. However, previous studies often combine semantic pairs with augmented positives or false neighbors (i.e., incorrectly mapped semantic pairs), making it difficult to isolate the effect of semantic pairing. We present a controlled empirical study of semantic positive pairs for self-supervised representation learning. From ImageNet-1K, we construct two matched subsets: an augmented-pair baseline and a manually curated semantic-pair dataset with the same class composition and training-pair count. We use these datasets to compare representative contrastive and non-contrastive SSL methods under matched training conditions. Across transfer learning and object detection evaluations, semantic-pair pretraining consistently improves generalisation over augmented-pair pretraining. Additional ablations show that semantic pairs induce invariances beyond the standard transformation pipeline. Among the evaluated methods, contrastive learning benefits most strongly from semantic pairs, with SimCLR showing the largest relative improvement. These results clarify the role of semantic positive pairs in SSL and provide guidance for selecting and designing frameworks that can exploit semantic pair information effectively
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。