arXiv:2503.15141cs.CV2025-03ICLR被引 7

首次实现无需冻结目标编码器的物体中心预训练,性能逼近海量数据训练模型。

Object-Centric Pretraining via Target Encoder Bootstrapping

  • 用动量更新的目标编码器自蒸馏,融合物体中心先验知识。
  • 在24.1万张COCO图像上训练,无监督物体发现性能媲美百万级图像模型。
  • 引入跨视图补丁筛选,防止初始目标编码器导致的槽塌陷问题。

物体中心表示学习近年来在真实世界数据集上取得成功,这得益于预训练的非物体中心基础模型作为槽注意力的重建目标。然而,目标编码器必须保持冻结,限制了物体中心模型的性能上限。通过自举更新目标编码器的尝试会导致性能大幅下降,原因在于其缺乏物体中心归纳偏置,导致物体中心模型编码器偏离有用的重建目标表示。为解决这一问题,我们提出一种全新的自蒸馏框架——物体中心预训练中的目标编码器自举(OCEBO),首次实现从零开始在真实数据上训练物体中心模型。在OCEBO中,目标编码器以指数移动平均方式由物体中心模型更新,从而显式融入槽注意力引入的物体中心归纳偏置,并打破原有性能上限。为缓解目标编码器随机初始化引发的槽塌陷问题,我们提出一种新型跨视图补丁过滤方法,仅对信息丰富的补丁提供监督。在24.1万张图像的COCO数据集上预训练后,OCEBO的无监督物体发现性能可与使用数亿图像预训练的非物体中心目标编码器模型相媲美。代码和预训练模型已公开于https://github.com/djukicn/ocebo。

原文摘要 · Abstract (English)

Object-centric representation learning has recently been successfully applied to real-world datasets. This success can be attributed to pretrained non-object-centric foundation models, whose features serve as reconstruction targets for slot attention. However, targets must remain frozen throughout the training, which sets an upper bound on the performance object-centric models can attain. Attempts to update the target encoder by bootstrapping result in large performance drops, which can be attributed to its lack of object-centric inductive biases, causing the object-centric model's encoder to drift away from representations useful as reconstruction targets. To address these limitations, we propose Object-CEntric Pretraining by Target Encoder BOotstrapping, a self-distillation setup for training object-centric models from scratch, on real-world data, for the first time ever. In OCEBO, the target encoder is updated as an exponential moving average of the object-centric model, thus explicitly being enriched with object-centric inductive biases introduced by slot attention while removing the upper bound on performance present in other models. We mitigate the slot collapse caused by random initialization of the target encoder by introducing a novel cross-view patch filtering approach that limits the supervision to sufficiently informative patches. When pretrained on 241k images from COCO, OCEBO achieves unsupervised object discovery performance comparable to that of object-centric models with frozen non-object-centric target encoders pretrained on hundreds of millions of images. The code and pretrained models are publicly available at https://github.com/djukicn/ocebo.

物体中心自蒸馏预训练槽注意力

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。