提出首个多视角物体变化检测基准与无需训练的方法。
SceneDiff: A Benchmark and Method for Multiview Object Change Detection
- 基于预训练模型，通过几何与语义特征对比检测变化
- 在多视角和双视角数据集上分别提升53%和30.6%的准确率
- 适用于跨场景、跨视角的物体增删移检测任务
我们研究在不同时间拍摄的同一场景图像或视频对中，识别新增、删除或移动的物体。准确识别可验证的变化极具挑战性——某些物体可能因遮挡或出框而看似消失，另一些则可能因视角大幅改变而外观不同。为此，我们引入了SceneDiff基准，这是首个基于不同相机轨迹拍摄的多视角变化检测数据集，包含350个多样化的视频对，具有密集的物体实例级标注。我们还提出了SceneDiff算法，一种无需训练的方法，通过求解图像位姿、分割图像为物体，并利用语义与几何特征进行比较。该方法依托预训练模型，无需重新训练即可跨领域泛化，且随基础模型升级而自然提升性能。在多视角和双视角基准上的实验表明，我们的方法分别相比现有方法提升了53.0%和30.6%的相对平均精度（AP）。
原文摘要 · Abstract (English)
We investigate the problem of identifying objects that have been added, removed, or moved between a pair of captures (images or videos) of the same scene at different times. Accurately identifying verifiable changes is extremely challenging -- some objects may appear to be missing because they are occluded or out of frame, while others may appear different due to large viewpoint changes. To study this problem, we introduce the SceneDiff Benchmark, the first multiview change detection dataset for scenes captured along different camera trajectories, comprising 350 diverse video pairs with dense object instance-level annotations. We also introduce the SceneDiff algorithm, a training-free approach that solves for image poses, segments images into objects, and compares them using semantic and geometric features. By building on pretrained models, SceneDiff generalizes across domains without retraining and naturally improves as the underlying models advance. Experiments on multiview and two-view benchmarks demonstrate that our method outperforms existing approaches by large margins (53.0\% and 30.6\% relative AP improvements). Project page: https://yuqunw.github.io/SceneDiff
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。