用视觉大模型和全局注意力提升场景变化检测的鲁棒性
Robust Scene Change Detection Using Visual Foundation Models and Cross-Attention Mechanisms
- 冻结DINOv2主干保留通用特征，用全图交叉注意力对齐图像差异
- 在VL-CMU-CD和PSCD上F1分数显著提升，尤其在几何变化场景下
- 适合需要抗光照、视角变化的遥感或视频监控应用
我们提出一种新型场景变化检测方法，利用视觉基础模型DINOv2的强特征提取能力，并引入全图交叉注意力机制，以应对光照变化、季节差异和视角差异等挑战。为有效学习图像对之间的对应与非对应关系，我们采用冻结主干网络以保持密集基础特征的通用性，并使用全图交叉注意力更好处理图像对间的视角差异。我们在两个基准数据集VL-CMU-CD和PSCD及其视角变异版本上进行评估。实验表明，该方法在涉及几何变化的场景中显著提升F1分数，相比现有最先进方法展现出更强的泛化能力，对光度和几何变化均具鲁棒性，且微调后能更好地适应新环境。详细的消融实验证实了架构中各组件的有效性。代码已开源：https://github.com/ChadLin9596/Robust-Scene-Change-Detection。
原文摘要 · Abstract (English)
We present a novel method for scene change detection that leverages the robust feature extraction capabilities of a visual foundational model, DINOv2, and integrates full-image cross-attention to address key challenges such as varying lighting, seasonal variations, and viewpoint differences. In order to effectively learn correspondences and mis-correspondences between an image pair for the change detection task, we propose to a) ``freeze'' the backbone in order to retain the generality of dense foundation features, and b) employ ``full-image'' cross-attention to better tackle the viewpoint variations between the image pair. We evaluate our approach on two benchmark datasets, VL-CMU-CD and PSCD, along with their viewpoint-varied versions. Our experiments demonstrate significant improvements in F1-score, particularly in scenarios involving geometric changes between image pairs. The results indicate our method's superior generalization capabilities over existing state-of-the-art approaches, showing robustness against photometric and geometric variations as well as better overall generalization when fine-tuned to adapt to new environments. Detailed ablation studies further validate the contributions of each component in our architecture. Our source code is available at: https://github.com/ChadLin9596/Robust-Scene-Change-Detection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。