用强化学习训练图像预训练模型，提升无标签数据表征能力
Visual Pre-Training on Unlabeled Images using Reinforcement Learning
- 将图像预训练建模为强化学习问题，通过视角变换和增强作为动作
- 在EpicKitchens、COCO、CC12M等数据集上获得更优特征表示
- 可通过弱标注或精选图片调节学习方向，灵活性强
在强化学习中，基于价值的算法会将每个观测与未来可能达到的状态和奖励关联起来。我们发现，许多自监督图像预训练方法与此范式相似：学习将图像裁块与邻近视图（如不同裁剪或颜色增强）关联。本文完成这一类比，提出将无标签图像数据（如网络爬取图像、视频帧）的预训练直接建模为强化学习问题。训练一个通用价值函数，在动态系统中，智能体通过改变视角或添加图像增强来转换图像。这种学习方式类似于裁块一致性自监督，但通过奖励函数提供简单调控机制，可利用精选图像或弱标注标题引导特征学习。实验表明，在真实世界无标签数据（包括EpicKitchens视频数据、COCO场景数据、CC12M网页爬取数据）上，该方法能生成更优表示。
原文摘要 · Abstract (English)
In reinforcement learning (RL), value-based algorithms learn to associate each observation with the states and rewards that are likely to be reached from it. We observe that many self-supervised image pre-training methods bear similarity to this formulation: learning features that associate crops of images with those of nearby views, e.g., by taking a different crop or color augmentation. In this paper, we complete this analogy and explore a method that directly casts pre-training on unlabeled image data like web crawls and video frames as an RL problem. We train a general value function in a dynamical system where an agent transforms an image by changing the view or adding image augmentations. Learning in this way resembles crop-consistency self-supervision, but through the reward function, offers a simple lever to shape feature learning using curated images or weakly labeled captions when they exist. Our experiments demonstrate improved representations when training on unlabeled images in the wild, including video data like EpicKitchens, scene data like COCO, and web-crawl data like CC12M.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。