无需标注数据,用大模型实现跨场景的无监督变化检测。
UniVCD: A New Method for Unsupervised Change Detection in the Open-Vocabulary Era
- 基于冻结的SAM2和CLIP,通过轻量对齐模块融合空间与语义信息。
- 在多个公开数据集上达到或超过现有方法的F1和IoU指标。
- 适合需要快速适配新场景且无标签数据的研究者使用。
变化检测(CD)从多时相观测中识别场景变化,广泛应用于城市规划与环境监测。现有方法多依赖有监督学习,性能受数据集限制且标注成本高,通常仅针对少数预定义类别,泛化能力差。随着SAM2和CLIP等视觉基础模型的发展,新的可能性出现。我们提出统一开放词汇变化检测(UniVCD),一种基于冻结SAM2和CLIP的无监督、开放词汇变化检测方法。UniVCD可在无任何标注数据或成对变化图像的情况下,检测跨多样化场景与成像几何的类别无关变化。引入轻量级特征对齐模块,桥接SAM2的空间细节表示与CLIP的语义先验,实现高分辨率、语义感知的变化估计,同时保持可训练参数极少。此外,设计简化后处理流程以抑制噪声和伪变化,提升边界清晰物体的检测精度。在多个公开的二值变化检测(BCD)和语义变化检测(SCD)基准上,UniVCD表现一致强劲,关键指标如F1和IoU达到或超越现有开放词汇方法。结果表明,利用冻结视觉基础模型与轻量多模态对齐进行无监督变化检测是一种可行且高效的开放词汇检测范式。代码与预训练模型将开源于https://github.com/Die-Xie/UniVCD。
原文摘要 · Abstract (English)
Change detection (CD) identifies scene changes from multi-temporal observations and is widely used in urban development and environmental monitoring. Most existing CD methods rely on supervised learning, making performance strongly dataset-dependent and incurring high annotation costs; they typically focus on a few predefined categories and generalize poorly to diverse scenes. With the rise of vision foundation models such as SAM2 and CLIP, new opportunities have emerged to relax these constraints. We propose Unified Open-Vocabulary Change Detection (UniVCD), an unsupervised, open-vocabulary change detection method built on frozen SAM2 and CLIP. UniVCD detects category-agnostic changes across diverse scenes and imaging geometries without any labeled data or paired change images. A lightweight feature alignment module is introduced to bridge the spatially detailed representations from SAM2 and the semantic priors from CLIP, enabling high-resolution, semantically aware change estimation while keeping the number of trainable parameters small. On top of this, a streamlined post-processing pipeline is further introduced to suppress noise and pseudo-changes, improving the detection accuracy for objects with well-defined boundaries. Experiments on several public BCD (Binary Change Detection) and SCD (Semantic Change Detection) benchmarks show that UniVCD achieves consistently strong performance and matches or surpasses existing open-vocabulary CD methods in key metrics such as F1 and IoU. The results demonstrate that unsupervised change detection with frozen vision foundation models and lightweight multi-modal alignment is a practical and effective paradigm for open-vocabulary CD. Code and pretrained models will be released at https://github.com/Die-Xie/UniVCD.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。