通过动态提升增广视图的熵，增强对比学习表征能力。
Maximizing Incremental Information Entropy for Contrastive Learning
- 设计可学习变换生成增量熵，优化增广视图间信息差异。
- 在小批量设置下，CIFAR-10/100、STL-10和ImageNet上性能提升显著。
- 模块可无缝融入现有框架，理论与实践结合紧密。
对比学习在自监督表征学习中取得显著成功，常以互信息最大化等信息论目标为导向。针对静态数据增强和刚性不变性约束的局限性，我们提出IE-CL（增量熵对比学习）框架，显式优化增广视图间的熵增，同时保持语义一致性。理论分析将编码器视为信息瓶颈，提出联合优化两个组件：用于熵生成的可学习变换和用于保留的编码器正则项。在CIFAR-10/100、STL-10和ImageNet上的实验表明，IE-CL在小批量设置下持续提升性能。核心模块可无缝集成至现有框架。该工作连接理论与实践，为对比学习提供了新视角。
原文摘要 · Abstract (English)
Contrastive learning has achieved remarkable success in self-supervised representation learning, often guided by information-theoretic objectives such as mutual information maximization. Motivated by the limitations of static augmentations and rigid invariance constraints, we propose IE-CL (Incremental-Entropy Contrastive Learning), a framework that explicitly optimizes the entropy gain between augmented views while preserving semantic consistency. Our theoretical framework reframes the challenge by identifying the encoder as an information bottleneck and proposes a joint optimization of two components: a learnable transformation for entropy generation and an encoder regularizer for its preservation. Experiments on CIFAR-10/100, STL-10, and ImageNet demonstrate that IE-CL consistently improves performance under small-batch settings. Moreover, our core modules can be seamlessly integrated into existing frameworks. This work bridges theoretical principles and practice, offering a new perspective in contrastive learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。