arXiv:2507.06979cs.LGcs.CV2025-07被引 2

提出新损失函数,让多视角对比学习更高效、更稳定。

A Principled Framework for Multi-View Contrastive Learning

  • 设计双新损失:同时建模所有视角间交互,避免目标冲突。
  • 在ImageNet等数据集上超越现有方法,支持五视以上仍有效。
  • 适用于多模态数据,解决维度坍缩问题,适合大规模预训练。

对比学习(CL)是自监督学习的主流范式,通常依赖于通过数据增强生成的成对视图。尽管在监督学习中多个增强视图能提升泛化能力,但现有对比学习方法对额外视图处理不佳,仅简单聚合成对目标。这带来四大缺陷:(L1) 每个样本使用多个优化项导致目标冲突;(L2) 无法建模所有视图与数据点间的交互;(L3) 继承了成对损失中的对齐-均匀性耦合问题;(L4) 无法充分发挥多视图在监督学习中展现的优势。本文提出两个新损失:MV-InfoNCE,将所有视图交互统一到单个项中;MV-DHEL,解耦视图间的对齐与均匀性,并随视图数量增长而扩展交互复杂度。二者均具理论基础,证明其渐近优化所有视图对齐与均匀性。实验证明,在ImageNet1K及三个其他数据集上,本方法持续优于现有方法,且能随视图增多而有效扩展。还将方法应用于多模态数据,显示其可超越两模态限制。消融实验表明,采用五视及以上时,MV-DHEL能有效缓解维度坍缩,充分使用嵌入空间,实现监督学习中观察到的多视图优势。

原文摘要 · Abstract (English)

Contrastive Learning (CL), a leading paradigm in Self-Supervised Learning (SSL), typically relies on pairs of data views generated through augmentation. While multiple augmentations per instance (more than two) improve generalization in supervised learning, current CL methods handle additional views suboptimally by simply aggregating different pairwise objectives. This approach suffers from four critical limitations: (L1) it utilizes multiple optimization terms per data point resulting to conflicting objectives, (L2) it fails to model all interactions across views and data points, (L3) it inherits fundamental limitations (e.g. alignment-uniformity coupling) from pairwise CL losses, and (L4) it prevents fully realizing the benefits of increased view multiplicity observed in supervised settings. We address these limitations through two novel loss functions: MV-InfoNCE, which extends InfoNCE to incorporate all possible view interactions simultaneously in one term per data point, and MV-DHEL, which decouples alignment from uniformity across views while scaling interaction complexity with view multiplicity. Both approaches are theoretically grounded - we prove they asymptotically optimize for alignment of all views and uniformity, providing principled extensions to multi-view contrastive learning. Our empirical results on ImageNet1K and three other datasets demonstrate that our methods consistently outperform existing multi-view approaches and effectively scale with increasing view multiplicity. We also apply our objectives to multimodal data and show that, in contrast to other contrastive objectives, they can scale beyond just two modalities. Most significantly, ablation studies reveal that MV-DHEL with five or more views effectively mitigates dimensionality collapse by fully utilizing the embedding space, thereby delivering multi-view benefits observed in supervised learning.

对比学习多视图自监督损失函数

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。