用物体级表征提升自监督图像模型的数据效率
Object-centric LeJEPA
- 以物体掩码为输入，将自监督学习从图像级升级到物体级
- 在10%-100% COCO数据上，各项任务性能均优于传统方法
- 适合追求小样本高效训练的计算机视觉研究者
基于LeJEPA的图像编码器可生成强表征用于下游任务，但与其他图像级自监督方法一样，通常需要大规模训练数据。在物体级别而非整个场景级别对齐表示有望提高数据效率，但完全自监督地实现这一目标——即同时进行场景分割和物体表征——存在不稳定性：分割依赖有意义的表示，而有意义的表示又依赖一致的分割。我们通过在训练中使用现成的SAM提议作为物体掩码来规避这一问题。我们将LeJEPA扩展为对物体级表征进行对齐，其分布反坍塌目标自然适用于变长物体集合。此外，引入实例分离损失，将同一场景中的其他物体视为负样本，进一步提升下游性能。在两种模型规模下，使用10%-100% COCO数据，物体级LeJEPA在跟踪（DAVIS）、分类（ImageNet-1k）、分割（ADE20k）和重识别（NAVI）任务上均优于图像级LeJEPA。
原文摘要 · Abstract (English)
Image encoders trained with LeJEPA can deliver strong features for downstream tasks, but, like other image-level self-supervised methods, typically require large training datasets. Aligning representations at the level of objects rather than whole scenes promises greater data efficiency, but doing this in a completely self-supervised way, effectively jointly partitioning a scene and representing its objects, is unstable: the two are locked in a cyclic dependency, partitioning requires meaningful representations, while meaningful representations require consistent partitioning. We sidestep this instability by taking object masks as given during training, using cheap, off-the-shelf SAM proposals. We extend LeJEPA - whose distributional anti-collapse objective ports naturally from whole images to variable-sized sets of objects - to align object-centric representations rather than whole images. An additional instance-separating loss, which treats other objects in the same scene as negatives, further boosts downstream performance. Across two model scales and 10-100% of COCO, object-level LeJEPA outperforms image-level LeJEPA on tracking (DAVIS), classification (ImageNet-1k), segmentation (ADE20k), and re-identification (NAVI).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。