自适应生成分层掩码,让自监督学习逐步捕捉从纹理到语义的视觉信息。
Evolved Hierarchical Masking for Self-Supervised Learning
- 用训练中的模型动态生成分层掩码,随训练阶段进化
- 在ImageNet-1K上比MAE高1.1%,ADE20K分割高1.4%
- 无需预训练或标注,适合低层细节与高层语义任务
现有掩码图像建模方法采用固定掩码模式引导自监督训练,因依赖不同标准刻画图像内容,固定模式限制了视觉线索建模能力。本文提出一种演进式分层掩码方法,旨在实现自监督学习中通用视觉线索建模。该方法利用正在训练的视觉模型将输入视觉线索解析为层次结构,并据此生成掩码。层次结构的准确性与所训练模型的能力相当,从而在不同训练阶段生成演进的掩码模式:初期聚焦低层视觉线索以捕捉基本纹理,随后逐步演化为描绘高层线索,强化对复杂对象语义和上下文的学习。该方法无需额外预训练模型或标注,通过动态调节训练难度保障效率。我们在七个下游任务上进行了广泛实验,包括依赖低层细节的局部重复图像检索,以及需要语义解析能力的图像分类和语义分割。实验结果表明,该方法显著提升各项任务性能。例如,在相同训练轮次下,其在ImageNet-1K分类上优于最近的MAE 1.1%,在ADE20K分割上优于MAE 1.4%。我们还将该方法与当前大语言模型研究趋势对齐,有效弥合了大规模预训练在语义密集型任务中的差距,并增强了对复杂细节感知能力要求的任务中的表现。
原文摘要 · Abstract (English)
Existing Masked Image Modeling methods apply fixed mask patterns to guide the self-supervised training. As those mask patterns resort to different criteria to depict image contents, sticking to a fixed pattern leads to a limited vision cues modeling capability.This paper introduces an evolved hierarchical masking method to pursue general visual cues modeling in self-supervised learning. The proposed method leverages the vision model being trained to parse the input visual cues into a hierarchy structure, which is hence adopted to generate masks accordingly. The accuracy of hierarchy is on par with the capability of the model being trained, leading to evolved mask patterns at different training stages. Initially, generated masks focus on low-level visual cues to grasp basic textures, then gradually evolve to depict higher-level cues to reinforce the learning of more complicated object semantics and contexts. Our method does not require extra pre-trained models or annotations and ensures training efficiency by evolving the training difficulty. We conduct extensive experiments on seven downstream tasks including partial-duplicate image retrieval relying on low-level details, as well as image classification and semantic segmentation that require semantic parsing capability. Experimental results demonstrate that it substantially boosts performance across these tasks. For instance, it surpasses the recent MAE by 1.1\% in imageNet-1K classification and 1.4\% in ADE20K segmentation with the same training epochs. We also align the proposed method with the current research focus on LLMs. The proposed approach bridges the gap with large-scale pre-training on semantic demanding tasks and enhances intricate detail perception in tasks requiring low-level feature recognition.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。