改进对比学习负样本筛选与优化，提升无监督表征质量
Momentum Contrastive Learning with Enhanced Negative Sampling and Hard Negative Filtering
- 设计双视角损失函数，平衡查询与关键嵌入优化
- 基于余弦相似度筛选难负样本，降低噪声干扰
- 在计算机视觉与自然语言处理中表现更优，适合追求高精度表征的场景
对比学习已成为无监督表示学习的关键范式，如动量对比（MoCo）等框架通过利用大规模负样本集提取判别性特征。然而，传统方法常忽视关键嵌入的潜力，且易受内存池中噪声负样本的影响而性能下降。本文提出一种增强型对比学习框架，包含两项核心创新：首先，引入双视角损失函数，确保查询与关键嵌入的平衡优化，提升表示质量；其次，设计选择性负样本采样策略，依据余弦相似度聚焦最具挑战性的难负样本，减轻噪声影响并增强特征判别力。大量实验表明，该框架在下游任务中表现优异，生成鲁棒且结构良好的表示。结果凸显优化对比机制在推进无监督学习及拓展其在计算机视觉与自然语言处理等领域应用方面的潜力。
原文摘要 · Abstract (English)
Contrastive learning has become pivotal in unsupervised representation learning, with frameworks like Momentum Contrast (MoCo) effectively utilizing large negative sample sets to extract discriminative features. However, traditional approaches often overlook the full potential of key embeddings and are susceptible to performance degradation from noisy negative samples in the memory bank. This study addresses these challenges by proposing an enhanced contrastive learning framework that incorporates two key innovations. First, we introduce a dual-view loss function, which ensures balanced optimization of both query and key embeddings, improving representation quality. Second, we develop a selective negative sampling strategy that emphasizes the most challenging negatives based on cosine similarity, mitigating the impact of noise and enhancing feature discrimination. Extensive experiments demonstrate that our framework achieves superior performance on downstream tasks, delivering robust and well-structured representations. These results highlight the potential of optimized contrastive mechanisms to advance unsupervised learning and extend its applicability across domains such as computer vision and natural language processing
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。