用视频动态训练图像模型,让模型关注物体本身而非背景。
VINO: Video-driven Invariance for Non-contextual Objects via Structural Prior Guided De-contextualization
- 通过结构先验生成视角,强制模型忽略背景干扰。
- 在PASCAL VOC上达到34.8的CorLoc,显著优于现有方法。
- 适合需要物体中心表征的视觉任务,如目标检测与分割。
自监督学习虽进展迅速,但特征常依赖背景纹理和共现统计等上下文捷径。视频虽具丰富时序变化,但密集真实场景中强自我运动导致前景与背景协同移动,使表示退化为场景编码器。为此,本文提出VINO(视频驱动的非上下文物体不变性),一种教师-学生框架,通过施加结构信息瓶颈,从密集视频中学习鲁棒图像编码器。利用类无关的结构先验生成视图(不作为语义伪标签),构建非对称知识蒸馏:教师从前景合并视图预测(背景抑制),学生观察保留上下文但去除竞争实例的对象条件场景视图。通过掩码蒸馏匹配目标,使背景线索不可靠,推动表示向物体中心不变性演进。进一步通过教师锚定的时间跨帧蒸馏强化物体持续性,并以掩码引导局部视图稳定部分与整体一致性。注意力可视化及在PASCAL VOC上的无监督对象发现实验表明,VINO有效解耦前景与背景。在稠密步行巡游威尼斯视频上预训练后,获得34.8的CorLoc,生成高度聚焦、形状敏感的表示,显著超越先前基于密集视频与运动引导的自监督基线。
原文摘要 · Abstract (English)
Self-supervised learning (SSL) has made rapid progress, yet learned features often over-rely on contextual shortcuts-background textures and co-occurrence statistics. While video provides rich temporal variation, dense in-the-wild streams with strong ego-motion create a co-occurrence trap: foreground objects and background context move coherently, encouraging representations to collapse into scene encoders. To address this, we propose VINO (Video-driven Invariance for Non-Contextual Objects), a teacher-student framework that learns robust image encoders from dense video by imposing a structural information bottleneck. Using a class-agnostic structural prior solely to generate views-not as semantic pseudo-labels-VINO forms an asymmetric distillation problem. The teacher predicts from a foreground-union view with the background suppressed, while the student observes object-conditioned scene views that retain surrounding context but remove competing instances. Matching these targets via masked distillation makes background cues unreliable, pushing the representation toward object-centric invariances. We further enforce temporal object permanence via teacher-anchored cross-time distillation over track-matched objects, and stabilize part-to-whole consistency with mask-guided local views. Through attention visualization and unsupervised object discovery on PASCAL VOC, we demonstrate that VINO effectively disentangles foreground from background. Pretrained on the dense Walking Tours Venice video, VINO achieves 34.8 CorLoc, yielding highly focused, shape-biased representations that substantially outperform prior dense-video and motion-guided SSL baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。