arXiv:2606.08332cs.CV2026-06

提出轻量级自监督学习方法SMI,降低计算成本同时提升性能。

SMI: Efficient Self-Supervised Learning via Mutual-Information-Inspired Dependency Optimization

论文配图:SMI: Efficient Self-Supervised Learning via Mutual-Information-Inspired Dependency Optimization
图 1 · 摘自论文原文
  • 基于高斯假设的互信息启发,优化样本级依赖矩阵。
  • 在ImageNet上达到顶尖性能,且计算复杂度显著降低。
  • 适合资源受限场景,尤其对细粒度数据集效果更优。

自监督学习(SSL)虽取得显著表征学习成果,但多数方法依赖大批次、内存库、动量编码器或全局同步机制,大幅增加计算成本与训练复杂度。本文提出语义互信息(SMI),一种基于高斯假设下互信息启发的依赖性公式推导出的轻量级自监督目标。与传统在高维特征相关矩阵上操作的相关性匹配目标不同,SMI通过非线性变换对成对相关性的样本级依赖矩阵进行优化,诱导出强调强语义关联对、同时保持表征多样性的独特优化动态。在使用ResNet-50主干网络的ImageNet上,SMI实现了与最先进SSL方法相当的线性评估性能,同时显著降低计算复杂度。在多个低资源基准测试中,SMI持续优于Barlow Twins,尤其在细粒度数据集上表现更佳。对优化动态和表征几何的分析表明,SMI具备更优的对齐-冗余平衡、更高的特征多样性以及更空间局部化的语义表征。结果表明,非线性依赖优化为传统基于相关性的自监督学习目标提供了一种高效且有效的替代方案。

原文摘要 · Abstract (English)

Self-supervised learning (SSL) has achieved remarkable representation learning performance, but many existing methods rely on large batch sizes, memory banks, momentum encoders, or global synchronization mechanisms that substantially increase computational cost and training complexity. In this work, we propose Semantic Mutual Information (SMI), a lightweight self-supervised objective derived from a mutual-information-inspired dependency formulation under Gaussian assumptions. Unlike conventional correlation matching objectives that operate on high-dimensional feature correlation matrices, SMI performs optimization on a sample-level dependency matrix through a nonlinear transformation of pairwise correlations. This formulation induces distinct optimization dynamics that emphasize strongly dependent semantic pairs while maintaining representation diversity. Experimental results on ImageNet using a ResNet-50 backbone demonstrate that SMI achieves competitive linear evaluation performance relative to state-of-the-art SSL approaches while substantially reducing computational complexity. Across multiple low-resource benchmarks, SMI consistently improves transfer performance over Barlow Twins, particularly on fine-grained datasets. Furthermore, analyses of optimization dynamics and representation geometry suggest improved alignment--redundancy balance, greater feature diversity, and more spatially localized semantic representations. These results indicate that nonlinear dependency optimization provides an effective and computationally efficient alternative to conventional correlation-based self-supervised learning objectives.

自监督学习互信息轻量化图像表征

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。