对比两种损失函数，发现三元组损失更利于保留细节和难例学习。
Comparing Contrastive and Triplet Loss: Variance Analysis and Optimization Behavior
- 通过方差分析与优化动态比较，揭示两类损失的内在差异。
- 三元组损失保持更高类内类间方差，提升表征细粒度。
- 适合关注难例与细节的任务，如图像检索与分类。
对比损失与三元组损失是深度度量学习中广泛应用的目标函数，但其对表征质量的影响仍不明确。本文通过理论与实证比较，聚焦类内与类间方差及优化行为（如贪婪更新）。在合成数据与真实数据集（MNIST、CIFAR-10）上采用一致设置进行任务实验，结果表明：三元组损失能更好保持类内与类间方差，支持更精细的表征区分；而对比损失倾向于压缩类内嵌入，可能掩盖细微语义差异。通过分析损失衰减速率、活跃比例与梯度范数，发现对比损失早期产生大量小更新，三元组损失则产生较少但更强的更新，持续作用于难例。在MNIST、CIFAR-10、CUB-200、CARS196上的分类与检索任务中，三元组损失始终表现更优，表明其更适合保留细节与聚焦难例，而对比损失则适用于平滑、全局的嵌入优化。
原文摘要 · Abstract (English)
Contrastive loss and triplet loss are widely used objectives in deep metric learning, yet their effects on representation quality remain insufficiently understood. We present a theoretical and empirical comparison of these losses, focusing on intra- and inter-class variance and optimization behavior (e.g., greedy updates). Through task-specific experiments with consistent settings on synthetic data and real datasets-MNIST, CIFAR-10-it is shown that triplet loss preserves greater variance within and across classes, supporting finer-grained distinctions in the learned representations. In contrast, contrastive loss tends to compact intra-class embeddings, which may obscure subtle semantic differences. To better understand their optimization dynamics, By examining loss-decay rate, active ratio, and gradient norm, we find that contrastive loss drives many small updates early on, while triplet loss produces fewer but stronger updates that sustain learning on hard examples. Finally, across both classification and retrieval tasks on MNIST, CIFAR-10, CUB-200, and CARS196 datasets, our results consistently show that triplet loss yields superior performance, which suggests using triplet loss for detail retention and hard-sample focus, and contrastive loss for smoother, broad-based embedding refinement.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。