通过预测潜在聚类提升图像自监督学习性能
Cluster and Predict Latent Patches for Improved Masked Image Modeling
- 用潜在特征聚类替代传统掩码重建，优化损失函数稳定性
- 在ImageNet上达83.8%准确率，ADE20K上32.1% mIoU
- 适合关注自监督视觉表征与模型架构创新的研究者
掩码图像建模（MIM）为自监督表示学习提供了有前景的路径，但现有MIM模型仍落后于最先进水平。本文系统分析了目标表示、损失函数与网络架构，提出一种纯MIM框架CAPI，其核心是预测潜在特征聚类。该方法采用基于聚类的损失，训练稳定且具备良好可扩展性。使用ViT-L骨干网络的CAPI在ImageNet上达到83.8%准确率，在ADE20K上实现32.1% mIoU，仅用简单线性探测器即显著优于此前MIM方法，接近当前最先进模型DINOv2的性能。代码与模型均已开源。
原文摘要 · Abstract (English)
Masked Image Modeling (MIM) offers a promising approach to self-supervised representation learning, however existing MIM models still lag behind the state-of-the-art. In this paper, we systematically analyze target representations, loss functions, and architectures, to introduce CAPI - a novel pure-MIM framework that relies on the prediction of latent clusterings. Our approach leverages a clustering-based loss, which is stable to train, and exhibits promising scaling properties. Our ViT-L backbone, CAPI, achieves 83.8% accuracy on ImageNet and 32.1% mIoU on ADE20K with simple linear probes, substantially outperforming previous MIM methods and approaching the performance of the current state-of-the-art, DINOv2. We release all our code and models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。