MAE通过自指导掩码提升图像表征学习效率
Self-Guided Masked Autoencoder
- 用自身聚类进度生成智能掩码，替代随机掩码
- 预训练早期即实现补丁级模式聚类，学习更高效
- 无需外部模型，保持自监督优势，适合视觉任务
掩码自编码器（MAE）是一种广泛应用于计算机视觉下游任务的自监督表示学习方法。尽管取得成功，但其学习机制仍不明确。本文通过深入分析发现，MAE在预训练初期即内在地学习基于模式的补丁级聚类。基于此认知，提出自指导掩码自编码器，利用自身补丁聚类进展生成有信息量的掩码，取代原始MAE中的随机掩码。该方法显著加速学习过程，且不依赖任何外部模型或附加信息，保持了MAE的自监督特性。在多个下游任务上的全面实验验证了该方法的有效性。
原文摘要 · Abstract (English)
Masked Autoencoder (MAE) is a self-supervised approach for representation learning, widely applicable to a variety of downstream tasks in computer vision. In spite of its success, it is still not fully uncovered what and how MAE exactly learns. In this paper, with an in-depth analysis, we discover that MAE intrinsically learns pattern-based patch-level clustering from surprisingly early stages of pretraining. Upon this understanding, we propose self-guided masked autoencoder, which internally generates informed mask by utilizing its progress in patch clustering, substituting the naive random masking of the vanilla MAE. Our approach significantly boosts its learning process without relying on any external models or supplementary information, keeping the benefit of self-supervised nature of MAE intact. Comprehensive experiments on various downstream tasks verify the effectiveness of the proposed method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。