在无约束拍摄条件下,实现室内场景的像素级变化检测。
VSCD: Video-based Scene Change Detection in Unaligned Scenes
- 以查询视频为中心,多参考帧动态匹配并融合变化特征
- 在超过110万帧数据上训练,达到当前最佳性能
- 适用于移动机器人视觉监控与增量学习,真实场景验证有效
环境变化检测对长期自主至关重要,但现有方法通常假设固定视角、轻微错位或仅有少量物体变化。本文提出视频级场景变化检测(VSCD),在参考视频与查询视频均来自同一室内空间但拍摄时间不同、相机运动不受限、无时间同步且物体可能增减的情况下,预测每个查询帧的像素级变化掩码。为此构建了一个包含超110万帧、标注精确变化掩码的大规模基准数据集,并提供真实世界测试集评估泛化能力。提出一种以查询为中心的多参考模型,通过变化掩码监督隐式学习时间匹配,利用局部块对应关系对齐候选参考特征,结合帧级和块级置信度融合各候选变化特征,最终每帧一次性解码高分辨率掩码。该方法在图像与视频基线中表现最优,并在移动机器人上成功部署于视觉监控与物体增量学习两个下游任务中。
原文摘要 · Abstract (English)
Detecting what has changed in an environment is essential for long-term autonomy, yet most change detection settings assume fixed viewpoints, mild misalignment, or only a few changed objects. We introduce Video-based Scene Change Detection (VSCD), which predicts a pixel-wise change mask for each query frame, given a reference and a query RGB video of the same indoor space recorded at different times under unconstrained camera motion. The two videos are not temporally synchronized, and many object instances may appear or disappear. To study this setting, we build a large-scale benchmark with over 1.1 million frames annotated with pixel-accurate change masks, together with a real-world test set for evaluating transfer beyond simulation. We propose a query-centric multi-reference model that learns temporal matching implicitly from change-mask supervision, aligns candidate reference features to the query via local patch correspondence, and fuses per-candidate change features using frame-level and patch-level confidence before decoding a high-resolution mask once per frame. Our approach achieves state-of-the-art performance against strong image- and video-based baselines, and we validate its real-world impact by deploying it on a mobile robot for two downstream applications -- visual surveillance and object incremental learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。