arXiv:2603.22758cs.CVcs.LG2026-03被引 1

解决视频对象学习中对象过度碎片化问题,通过渐进式分配槽位提升表现。

Reconstruction-Guided Slot Curriculum: Addressing Object Over-Fragmentation in Video Object-Centric Learning

  • 从少量粗粒度槽位开始,按重建误差逐步添加新槽位,避免冗余分裂。
  • 引入结构感知损失,增强槽位语义边界清晰度,减少模糊分割。
  • 采用循环推理机制,实现帧间时序一致性,适合早期帧建模。

视频对象中心学习旨在将原始视频分解为少量对象槽,但现有槽注意力模型常出现严重过度碎片化问题。这是因为模型隐式倾向于占用所有槽以最小化重建目标,导致单个对象被多个冗余槽表示。为此,本文提出重建引导的槽位课程(SlotCurri):训练初期仅使用少量粗粒度槽位,随后在重建误差高的区域逐步分配新槽,仅在需要时扩展容量,从源头防止碎片化。然而,在槽位扩展过程中,有意义的子部分仅在粗粒度语义已充分分离时才能形成;而初始槽预算少且使用MSE目标时,语义边界仍模糊。因此,本文在MSE基础上增加结构感知损失,保留局部对比度和边缘信息,促进槽位语义边界锐化。最后,提出循环推理机制,使槽位在帧序列中正向与反向传播,生成即使在早期帧也具时序一致性的对象表征。整体上,SlotCurri通过在重建失败处分配容量,并结合结构线索与循环推理,有效缓解对象过度碎片化。在YouTube-VIS和MOVi-C上的显著性能提升验证了其有效性,分别获得+6.8和+8.3的FG-ARI增益。代码已开源。

原文摘要 · Abstract (English)

Video Object-Centric Learning seeks to decompose raw videos into a small set of object slots, but existing slot-attention models often suffer from severe over-fragmentation. This is because the model is implicitly encouraged to occupy all slots to minimize the reconstruction objective, thereby representing a single object with multiple redundant slots. We tackle this limitation with a reconstruction-guided slot curriculum (SlotCurri). Training starts with only a few coarse slots and progressively allocates new slots where reconstruction error remains high, thus expanding capacity only where it is needed and preventing fragmentation from the outset. Yet, during slot expansion, meaningful sub-parts can emerge only if coarse-level semantics are already well separated; however, with a small initial slot budget and an MSE objective, semantic boundaries remain blurry. Therefore, we augment MSE with a structure-aware loss that preserves local contrast and edge information to encourage each slot to sharpen its semantic boundaries. Lastly, we propose a cyclic inference that rolls slots forward and then backward through the frame sequence, producing temporally consistent object representations even in the earliest frames. All combined, SlotCurri addresses object over-fragmentation by allocating representational capacity where reconstruction fails, further enhanced by structural cues and cyclic inference. Notable FG-ARI gains of +6.8 on YouTube-VIS and +8.3 on MOVi-C validate the effectiveness of SlotCurri. Our code is available at github.com/wjun0830/SlotCurri.

视频理解对象中心槽位学习时序一致性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。