揭示对比学习的几何机制，解释跨模态差异如何形成。
The Geometric Mechanics of Contrastive Representation Learning: Alignment Potentials, Entropic Dispersion, and Cross-modal Divergence
- 用测度论框架分析表示分布的演化，揭示能量景观的几何分岔。
- 在多模态情况下，熵与对称性导致持久的模态差距。
- 适用于研究对比学习、跨模态对齐和表征几何的学者。
尽管InfoNCE是现代对比学习的基础，但其几何机制仍缺乏深入刻画，仅限于典型的对齐-均匀性分解。本文构建了一个测度论框架，其中表示测度在固定嵌入流形上演化。在大批次极限下，我们证明了值和梯度的一致性，将随机目标与明确的确定性能量景观联系起来，并揭示了单峰与对称多峰态之间的几何分岔。在单峰情形中，内在能量严格凸，存在唯一吉布斯平衡，表明熵在对齐区域内起决定性作用。在多峰情形中，内在几何变为交叉耦合，包含持续存在的负对称发散项：每种模态的边缘分布重塑另一种的有效景观，使得强成对对齐与持续的模态差距共存。受控的合成实验及预训练CLIP表示的分析支持这些预测。总体而言，我们的结果将分析视角从点对点判别转向群体几何，表明仅靠成对对齐无法控制跨模态边缘结构。
原文摘要 · Abstract (English)
While InfoNCE underlies modern contrastive learning, its geometric mechanisms remain under-characterized beyond the canonical alignment--uniformity decomposition. We develop a measure-theoretic framework in which representation measures evolve on a fixed embedding manifold. In the large-batch limit, we prove value and gradient consistency, linking the stochastic objective to explicit deterministic energy landscapes and revealing a geometric bifurcation between unimodal and symmetric multimodal regimes. In the unimodal case, the intrinsic energy is strictly convex and admits a unique Gibbs equilibrium, showing that entropy acts as a tie-breaker within the aligned basin. In the multimodal case, the intrinsic geometry becomes cross-coupled and contains a persistent negative symmetric divergence term: each modality's marginal reshapes the effective landscape of the other, allowing strong pairwise alignment to coexist with a persistent modality gap. Controlled synthetic experiments and analyses of pretrained CLIP representations support these predictions. Overall, our results shift the analytical lens from pointwise discrimination to population geometry, showing that pairwise alignment alone is insufficient to control cross-modal marginal structure.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。