提出新型解码器让视频对象槽位摆脱位置束缚，提升动态物体分割性能。
Semantic Slots for Video Object-Centric Learning
- 用基于Transformer的解码器让槽位成为语义查询，不再绑定空间位置。
- 单帧计算的槽位可分解后续视频帧，无需复杂时序预测或辅助损失。
- 在YouTube-VIS上达到86.6% ARI、62.8% mBO，领先现有方法21点。
视频对象中心学习（OCL）传统上聚焦于优化编码器以保证时序一致性。本文认为主要瓶颈在于解码器。传统解码器强制槽位空间锚定，限制其对运动的适应能力。为此提出SemanticSlots，采用基于Transformer的解码器，利用图像上下文信息，使槽位无需编码边界精度和空间坐标。这使得槽位作为语义查询，具备对象位置不变性，能检索匹配特征而非记忆坐标。更重要的是，单帧生成的槽位可直接分解后续视频帧，无需复杂时序预测或辅助时序损失。在YouTube-VIS数据集上，SemanticSlots相比VideoSAUR提升31点mBO，优于当前最先进方法21点，达86.6% ARI和62.8% mBO。
原文摘要 · Abstract (English)
Video Object-Centric Learning (OCL) has traditionally focused on refining the encoder architecture to ensure temporal consistency. In this paper, we argue that the primary bottleneck lies in the decoder. We show that traditional decoders force slots to be spatially anchored, hindering their ability to adapt to motion. We propose SemanticSlots, which uses a Transformer-based decoder that leverages image context, relieving slots from encoding boundary precision and spatial location. This allows slots to function as semantic queries that are inherently object position invariant, retrieving matching features rather than memorizing coordinates. More importantly, this property allows slots computed from a single frame to decompose subsequent video frames, eliminating the need for complex temporal predictors or auxiliary temporal losses. Results on YouTube-VIS show that SemanticSlots improves upon VideoSAUR by 31 points in mBO and outperforms current state-of-the-art methods by 21 points, achieving 86.6% ARI and 62.8% mBO.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。