让视觉模型自动适应物体数量变化,突破固定槽位限制。
MetaSlot: Break Through the Fixed Number of Slots in Object-Centric Learning
- 用向量量化构建物体原型库,动态匹配不同数量的物体
- 消除重复槽位,使每个物体仅对应一个槽,提升可解释性
- 渐进加噪加速收敛,适配现有模型无需重构
学习对象级、结构化的表示被认为是提升视觉模型泛化能力的关键,并支撑下一代预训练视觉模型(PVMs)的设计。主流对象中心学习(OCL)方法采用槽注意力(Slot Attention)或其变体,通过迭代聚合对象的超像素到一组固定数量的查询特征向量(称为槽)。然而,依赖固定槽数会导致物体数量变化时,同一物体被拆分为多个部分。我们提出MetaSlot,一种可即插即用的槽注意力变体,能适应可变的物体数量。MetaSlot (i) 通过向量量化将槽表示聚类,建立数据集中物体的原型代码本;(ii) 利用该代码本对传统聚合得到的槽进行量化,移除重复槽;(iii) 在槽注意力迭代中逐步注入更弱的噪声,以加速并稳定聚合过程。MetaSlot是通用的槽注意力变体,可无缝集成至现有OCL架构中。在多个公开数据集和任务(包括物体发现与识别)上,配备MetaSlot的模型相比现有槽注意力变体,显著提升性能并产生更易解释的槽表示。
原文摘要 · Abstract (English)
Learning object-level, structured representations is widely regarded as a key to better generalization in vision and underpins the design of next-generation Pre-trained Vision Models (PVMs). Mainstream Object-Centric Learning (OCL) methods adopt Slot Attention or its variants to iteratively aggregate objects' super-pixels into a fixed set of query feature vectors, termed slots. However, their reliance on a static slot count leads to an object being represented as multiple parts when the number of objects varies. We introduce MetaSlot, a plug-and-play Slot Attention variant that adapts to variable object counts. MetaSlot (i) maintains a codebook that holds prototypes of objects in a dataset by vector-quantizing the resulting slot representations; (ii) removes duplicate slots from the traditionally aggregated slots by quantizing them with the codebook; and (iii) injects progressively weaker noise into the Slot Attention iterations to accelerate and stabilize the aggregation. MetaSlot is a general Slot Attention variant that can be seamlessly integrated into existing OCL architectures. Across multiple public datasets and tasks--including object discovery and recognition--models equipped with MetaSlot achieve significant performance gains and markedly interpretable slot representations, compared with existing Slot Attention variants.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。