分离物体外观与姿态,提升视频目标追踪的稳定性和分割精度
Rethinking Object-Centric Representations for Video Dynamics Modeling

- 将每个槽位分解为外观和几何姿态,解耦变化因素
- 在真实与合成数据上显著优于现有方法,尤其在遮挡和进出场景中
- 自适应门控机制动态调节活跃槽数,防止过度分割
无监督视频对象追踪旨在不依赖人工标注的情况下,将动态场景分解为持续存在的、以对象为中心的实体。许多近期方法采用基于槽位的表示,用一组固定的潜在变量(“槽位”)跨帧表示单个对象。为保持对象身份一致性,这些模型对槽位嵌入施加时间一致性约束。然而,当外观与姿态纠缠时,该一致性目标会与对象运动和视角变化冲突,导致槽位倾向于锁定静态区域(如背景),而前景对象则被分散到多个槽位或频繁更换身份。为此,我们提出STAITUS,一种统一框架,显式将每个槽位解耦为外观和几何姿态(位置/尺度)。利用这一解耦,STAITUS在帧内强制空间分离,并仅在外观空间施加时间对齐,从而在运动、遮挡和对象出入条件下获得更清晰的掩码和更持久的身份。此外,为缓解过度分割问题,引入自适应门控机制,动态调整活跃槽数以匹配场景复杂度。在合成与真实世界基准上的大量实验表明,STAITUS在分割质量与追踪稳定性方面显著优于现有最优基线。
原文摘要 · Abstract (English)
Unsupervised video object tracking aims to decompose dynamic scenes into persistent, object-centric entities without manual annotations. Many recent approaches rely on slot-based representations, where a fixed set of latent variables ("slots") represent individual objects across frames. To preserve object identity, these models enforce temporal consistency on slot embeddings. However, when appearance and pose are entangled, this consistency objective conflicts with object motion and viewpoint changes. As a result, slots tend to lock onto static regions (e.g., background) to satisfy the consistency objective, while foreground objects become fragmented across multiple slots or frequently swap identities. To address these limitations, we propose STAITUS, a unified framework that explicitly disentangles each slot into appearance and geometric pose (position/scale). Leveraging this disentanglement, STAITUS enforces within-frame spatial separation and applies temporal alignment only in appearance space, yielding sharper masks and more persistent identities under motion, occlusion, and object entry/exit. Furthermore, to mitigate over-segmentation, we introduce an adaptive gating mechanism that dynamically adjusts the number of active slots to match scene complexity. Extensive experiments on synthetic and real-world benchmarks demonstrate that STAITUS substantially outperforms state-of-the-art baselines in segmentation quality and tracking stability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。