用遮蔽图像补全训练大模型，性能超越有监督预训练。
Masked Autoencoders Are Scalable Vision Learners
- 仅用可见图像块训练编码器，解码器重建被遮蔽区域。
- 遮蔽75%图像块仍能学出高质量特征，准确率达87.8%。
- 适合大规模视觉模型训练，尤其适合无标签数据场景。
本文表明，掩码自编码器（MAE）是可扩展的计算机视觉自监督学习方法。其核心思想是随机遮蔽输入图像的局部区域，并重建缺失像素。该方法基于两个关键设计：首先，采用非对称编码器-解码器结构，编码器仅处理可见图像块（不包含遮蔽标记），解码器则通过潜在表示和遮蔽标记重构原图；其次，发现高比例遮蔽（如75%）仍能形成有意义的自监督任务。结合这两项设计，可高效训练大型模型：训练速度提升3倍以上，精度显著提高。该方法支持训练高容量模型并实现良好泛化能力，例如在仅使用ImageNet-1K数据的情况下，一个标准ViT-Huge模型达到87.8%最高准确率，下游任务迁移性能优于有监督预训练，并展现出良好的可扩展性。
原文摘要 · Abstract (English)
This paper shows that masked autoencoders (MAE) are scalable self-supervised learners for computer vision. Our MAE approach is simple: we mask random patches of the input image and reconstruct the missing pixels. It is based on two core designs. First, we develop an asymmetric encoder-decoder architecture, with an encoder that operates only on the visible subset of patches (without mask tokens), along with a lightweight decoder that reconstructs the original image from the latent representation and mask tokens. Second, we find that masking a high proportion of the input image, e.g., 75%, yields a nontrivial and meaningful self-supervisory task. Coupling these two designs enables us to train large models efficiently and effectively: we accelerate training (by 3x or more) and improve accuracy. Our scalable approach allows for learning high-capacity models that generalize well: e.g., a vanilla ViT-Huge model achieves the best accuracy (87.8%) among methods that use only ImageNet-1K data. Transfer performance in downstream tasks outperforms supervised pre-training and shows promising scaling behavior.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。