自监督ViT能直接提取图像语义分割信息，效果优于传统网络。
Emerging Properties in Self-Supervised Vision Transformers
- 用无标签数据训练的ViT可显式学习图像语义分割特征
- 小规模ViT在ImageNet上达78.3%准确率，作k-NN分类器表现优异
- 提出DINO方法，结合动量编码器与多裁剪训练，适合自监督场景
本文探讨自监督学习是否为视觉变换器（ViT）带来超越卷积网络（convnets）的新特性。除了自监督方法在该架构上表现良好外，我们发现：第一，自监督ViT特征包含清晰的图像语义分割信息，而监督式ViT或convnets中不明显；第二，这些特征作为k-NN分类器时，在ImageNet上达到78.3%的top-1准确率。研究还强调了动量编码器、多裁剪训练及小块输入对ViT的重要性。我们基于这些发现构建了一个简单自监督方法——DINO，可视为无标签的自蒸馏过程。通过该方法，ViT-Base在ImageNet线性评估中达到80.1% top-1准确率，展现出显著的协同效应。
原文摘要 · Abstract (English)
In this paper, we question if self-supervised learning provides new properties to Vision Transformer (ViT) that stand out compared to convolutional networks (convnets). Beyond the fact that adapting self-supervised methods to this architecture works particularly well, we make the following observations: first, self-supervised ViT features contain explicit information about the semantic segmentation of an image, which does not emerge as clearly with supervised ViTs, nor with convnets. Second, these features are also excellent k-NN classifiers, reaching 78.3% top-1 on ImageNet with a small ViT. Our study also underlines the importance of momentum encoder, multi-crop training, and the use of small patches with ViTs. We implement our findings into a simple self-supervised method, called DINO, which we interpret as a form of self-distillation with no labels. We show the synergy between DINO and ViTs by achieving 80.1% top-1 on ImageNet in linear evaluation with ViT-Base.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。