用合成数据和假难例训练视觉模型，提升自监督学习效果
Fake & Square: Training Self-Supervised Vision Transformers with Synthetic Data and Synthetic Hard Negatives
- 用生成模型创造合成数据增强多样性
- 在特征空间生成合成难例构建挑战性对比
- 适合研究自监督学习与数据增广的学者
本文不提出新方法，而是基于现有自监督学习范式，借鉴'假装直到成功'的理念。尽管对比学习已取得显著进展，但通常依赖大量真实数据和精心设计的难例。为此，我们探索视觉变压器中的两种'伪造'方式：一是利用生成模型创建合成数据以增强样本多样性；二是尝试在表示空间生成合成难例，构造更具挑战性的对比。所提出的框架Syn2Co结合二者，在DeiT-S和Swin-T架构上评估合成增强训练对视觉表征鲁棒性和可迁移性的提升效果。结果表明，合成数据在自监督学习中具有潜力但也存在局限，为未来研究提供重要启示。
原文摘要 · Abstract (English)
This paper does not introduce a new method per se. Instead, we build on existing self-supervised learning approaches for vision, drawing inspiration from the adage "fake it till you make it". While contrastive self-supervised learning has achieved remarkable success, it typically relies on vast amounts of real-world data and carefully curated hard negatives. To explore alternatives to these requirements, we investigate two forms of "faking it" in vision transformers. First, we study the potential of generative models for unsupervised representation learning, leveraging synthetic data to augment sample diversity. Second, we examine the feasibility of generating synthetic hard negatives in the representation space, creating diverse and challenging contrasts. Our framework - dubbed Syn2Co - combines both approaches and evaluates whether synthetically enhanced training can lead to more robust and transferable visual representations on DeiT-S and Swin-T architectures. Our findings highlight the promise and limitations of synthetic data in self-supervised learning, offering insights for future work in this direction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。