arXiv:2606.12601cs.CV2026-06

分离物体外观与身份,提升视频目标识别的稳定性。

Dual-State Slot Attention: Decoupling Appearance and Identity for Video Object-Centric Learning

论文配图:Dual-State Slot Attention: Decoupling Appearance and Identity for Video Object-Centric Learning
图 1 · 摘自论文原文
  • 将每个槽分为外观态和身份态,分别处理视觉变化与时间一致性。
  • 在MOVi-C、MOVi-D等数据集上,分割精度与时序一致性显著提升。
  • 适合需要稳定目标追踪的视频理解任务,如动态预测与识别。

无监督视频目标中心学习旨在将动态场景分解为持久的目标级表征,而无需人工标注。然而,现有基于槽的方法在快速运动和部分遮挡等挑战性场景下难以维持稳定的目标身份。首先,它们通常将每帧的外观信息与跨帧的身份信息编码在同一槽向量中,造成目标冲突:重建需要对瞬时视觉变化敏感,而时序一致性则要求对这些变化不变。其次,槽注意力中的令牌归一化会放大弱匹配槽的更新,使其吸收其他物体的令牌,破坏槽与物体的对应关系。我们提出双状态槽注意力(DSSA),一种完全自监督的框架,通过分离外观与身份,并减少弱匹配槽的虚假更新来解决这些问题。DSSA将每个槽分解为局部状态(用于每帧外观)和身份状态(用于时序稳定的物体信息),从而分别实现重建与时序一致性的优化。身份状态通过学习的递归转移进行更新,作为局部状态的时间滤波器;竞争调节聚合(CMA)则抑制弱匹配槽的更新,防止其吸收其他物体的令牌。在MOVi-C、MOVi-D和YouTube-VIS上的实验表明,DSSA在分割质量与时序一致性方面持续优于先前方法,同时在下游目标识别与视频动态预测任务中表现更优。代码与模型将在论文录用后公开。

原文摘要 · Abstract (English)

Unsupervised video object-centric learning aims to decompose dynamic scenes into persistent, object-level representations without supervision. However, existing slot-based methods struggle to maintain stable object identity in challenging settings such as rapid motion and partial occlusion. First, they typically encode both the per-frame appearance of an object and its identity across frames in a single slot vector, creating an objective conflict that leads to slot swapping: reconstruction requires sensitivity to transient visual changes, whereas temporal consistency requires invariance to them. Second, the token renormalization used in Slot Attention can amplify weakly attending slots, allowing them to absorb tokens from other objects and destabilize slot-to-object correspondence. We propose Dual-State Slot Attention (DSSA), a fully self-supervised framework that addresses these limitations by separating appearance from identity and by reducing spurious updates from weakly matching slots. DSSA decomposes each slot into a local state for per-frame appearance and an identity state for temporally stable object information, thereby aligning reconstruction and temporal consistency with separate representations. The identity state is updated through a learned recurrent transition that acts as a temporal filter on the local state, while competition-modulated aggregation (CMA) down-weights updates from weakly matching slots and prevents them from absorbing tokens from other objects. Experiments on MOVi-C, MOVi-D, and YouTube-VIS demonstrate that DSSA consistently improves segmentation quality and temporal consistency over prior methods, while also yielding stronger downstream object recognition and video dynamics prediction. Code and models will be made publicly available upon acceptance.

视频生成注意力机制目标追踪

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。