通过自学习二值掩码，高效适配大模型到少标注任务。
Self-Masking Networks for Unsupervised Adaptation
- 用自监督方式学习二值掩码，替代传统参数微调。
- 存储效率提升79倍，少样本场景下性能显著提升。
- 适用于视觉领域少标注任务，尤其适合资源受限场景。
随着千亿参数基础模型的出现，高效微调在将模型适配至下游任务中变得愈发重要。然而，在计算机视觉领域，当缺乏高质量标注数据时，仍难以获得良好性能。本文提出一种自监督方法，通过学习二值掩码来适配预训练通用模型。这些自监督掩码网络（SMNs）存储效率最高可达79倍，并显著提升在标签高效下游任务中的表现。我们在8个数据集和3种模型架构上验证了学习二值掩码作为微调方法的有效性，并在3种标签稀缺场景中展示了SMNs的优越性。
原文摘要 · Abstract (English)
With the advent of billion-parameter foundation models, efficient fine-tuning has become increasingly important for the adaptation of models to downstream tasks. However, especially in computer vision, it can be hard to achieve good performance when access to quality labeled data is lacking. In this work, we propose a method adapting pretrained generalist models in a self-supervised manner by learning binary masks. These self-supervised masking networks (SMNs) are up to 79x more efficient to store and significantly improve performance on label-efficient downstream tasks. We validate the usefulness of learning binary masks as a fine-tuning method on 8 datasets and 3 model architectures, and we demonstrate the effectiveness of SMNs in 3 label-efficient settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。