不用文本编码器，直接用原始文本做分类标签。
Classification Done Right for Vision-Language Pre-Training
- 用分词后的原始文本作监督标签，省去文本筛选和编码
- 在多个视觉语言任务上表现优于CLIP，且无需大批次训练
- 适合追求简单高效、资源受限的视觉语言预训练场景
我们提出SuperClass，一种用于图像-文本数据视觉语言预训练的超简单分类方法。与需要文本编码器作为对比目标的CLIP不同，SuperClass直接使用分词后的原始文本作为监督分类标签，无需额外文本筛选或选择。由于不依赖文本编码器作为对比目标，SuperClass无需文本编码器，也无需维持大批次训练。SuperClass在多种下游任务中表现出色，包括经典计算机视觉基准和视觉语言任务。我们进一步研究了模型规模、训练长度和数据量对SuperClass的扩展行为，报告了令人鼓舞的结果，并与CLIP进行了对比。
原文摘要 · Abstract (English)
We introduce SuperClass, a super simple classification method for vision-language pre-training on image-text data. Unlike its contrastive counterpart CLIP who contrast with a text encoder, SuperClass directly utilizes tokenized raw text as supervised classification labels, without the need for additional text filtering or selection. Due to the absence of the text encoding as contrastive target, SuperClass does not require a text encoder and does not need to maintain a large batch size as CLIP does. SuperClass demonstrated superior performance on various downstream tasks, including classic computer vision benchmarks and vision language downstream tasks. We further explored the scaling behavior of SuperClass on model size, training length, or data size, and reported encouraging results and comparisons to CLIP. https://github.com/x-cls/superclass
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。