无需标注数据，通过运动信息自动发现视频中多个物体。
Motion-Refined DINOSAUR for Unsupervised Multi-Object Discovery
- 利用无相机运动的视频帧和自监督光流做运动分割生成伪标签。
- 在TRI-PD和KITTI数据集上超越已有最先进方法，完全无监督。
- 结构简单但有效，适合需要自动目标发现的研究场景。
无监督多物体发现（MOD）旨在不依赖任何人工标注的情况下，从视觉场景中检测并定位不同物体实例。近期方法利用物体中心学习（OCL）和视频中的运动线索识别个体物体，但通常需借助监督生成伪标签来训练模型。本文提出MR-DINOSAUR——一种极简的无监督方法，扩展自预训练的自监督物体中心模型DINOSAUR，用于无监督多物体发现。通过检索无相机运动的视频帧，并对无监督光流进行运动分割，生成高质量的无监督伪标签。利用这些伪标签优化DINOSAUR的槽位表示，并引入槽位去激活模块，将槽位分配给前景与背景。尽管方法概念简洁，MR-DINOSAUR在TRI-PD和KITTI数据集上表现优异，超越先前最优结果，且全程无监督。
原文摘要 · Abstract (English)
Unsupervised multi-object discovery (MOD) aims to detect and localize distinct object instances in visual scenes without any form of human supervision. Recent approaches leverage object-centric learning (OCL) and motion cues from video to identify individual objects. However, these approaches use supervision to generate pseudo labels to train the OCL model. We address this limitation with MR-DINOSAUR -- Motion-Refined DINOSAUR -- a minimalistic unsupervised approach that extends the self-supervised pre-trained OCL model, DINOSAUR, to the task of unsupervised multi-object discovery. We generate high-quality unsupervised pseudo labels by retrieving video frames without camera motion for which we perform motion segmentation of unsupervised optical flow. We refine DINOSAUR's slot representations using these pseudo labels and train a slot deactivation module to assign slots to foreground and background. Despite its conceptual simplicity, MR-DINOSAUR achieves strong multi-object discovery results on the TRI-PD and KITTI datasets, outperforming the previous state of the art despite being fully unsupervised.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。