为自监督学习设计多任务潜在空间对齐，提升模型性能
Self-Supervised Learning with a Multi-Task Latent Space Objective
- 为每种图像变换分配独立预测器，避免任务冲突
- 在ImageNet和COCO上实现3.8%-4%的线性评估性能提升
- 适合想改进自监督预训练的视觉模型研究者
我们提出一种多任务自预测孪生自监督学习方法，其中每个空间变换定义一个独立的潜在空间对齐任务，由共享编码器上的专用预测器解决。这一视角解释了以往多裁剪训练在BYOL、SimSiam和MoCo v3等方法中的长期失败原因：共享预测器需同时处理异构对齐任务，导致优化不稳定。为每种视图类型分配专属预测器可解决此干扰，使各框架在线性评估中性能提升3.8%-4%。该视角还提供了一种有原则的方法来丰富预训练，通过引入额外空间变换作为互补任务。我们以非对称遮蔽视图为例，即掩码的在线视图与完整的目标视图对齐，形成语义补全目标。所提框架稳定、不依赖主干网络，且持续提升了ResNet和ViT在ImageNet和COCO上的表现。
原文摘要 · Abstract (English)
We propose a multi-task formulation of self-predictive Siamese SSL in which each spatial transformation defines a distinct latent-space alignment task, solved by a dedicated predictor over a shared encoder. This perspective directly explains a long-standing failure of multi-crop training in self-predictive methods such as BYOL, SimSiam, and MoCo v3: a shared predictor is forced to solve heterogeneous alignment tasks simultaneously, leading to unstable optimization. Assigning one predictor per view type resolves this interference, unlocking linear evaluation gains of 3.8-4\% across frameworks. This perspective also suggests a principled way to enrich pre-training by introducing additional spatial transformations as complementary tasks. We demonstrate this by introducing asymmetric cutout views, in which a masked online view is aligned with a complete target, forming a semantic inpainting objective. The resulting framework is stable, backbone-agnostic, and consistently improves the performance of ResNet and ViT models on ImageNet and COCO.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。