用文本提示统一定义场景变化,让模型跨数据集通用。
ViewDelta: Scaling Scene Change Detection through Text-Conditioning
- 用自然语言描述变化类型,精准区分重要与无关变动。
- 单模型在5个数据集上训练,性能超越专用模型。
- 发布超50万图像对的合成数据集,支持文本条件检测。
我们提出ViewDelta,一种基于文本条件的场景变化检测框架,解决不同数据集中“相关变化”与“干扰变化”标注不一致的歧义问题。现有方法因数据集标签差异难以泛化,如植被生长或车道线变化在某些数据集中被视为相关,另一些则视为无关。ViewDelta通过自然语言提示精确指定相关变化,如单一属性、特定类别集合或所有可观察差异。为支持该范式,我们发布了首个大规模合成数据集CSeg,包含超过50万对图像和30多万条唯一文本提示。实验表明,一个在CSeg、SYSU-CD、PSCD、VL-CMU-CD及其非对齐变体上联合训练的ViewDelta模型,在性能上达到或超过各数据集专用模型水平,验证了文本条件在实现可泛化变化检测中的有效性。代码与数据集见https://joshuakgao.github.io/viewdelta/。
原文摘要 · Abstract (English)
We introduce a generalized framework for Scene Change Detection (SCD) that addresses the core ambiguity of distinguishing "relevant" from "nuisance" changes, enabling effective joint training of a single model across diverse domains and applications. Existing methods struggle to generalize due to differences in dataset labeling, where changes such as vegetation growth or lane marking alterations may be labeled as relevant in one dataset and irrelevant in another. To resolve this ambiguity, we propose ViewDelta, a text conditioned change detection framework that uses natural language prompts to define relevant changes precisely, such as a single attribute, a specific set of classes, or all observable differences. To facilitate training in this paradigm, we release the Conditional Change Segmentation dataset (CSeg), the first large-scale synthetic dataset for text conditioned SCD, consisting of over 500,000 image pairs with more than 300,000 unique textual prompts describing relevant changes. Experiments demonstrate that a single ViewDelta model trained jointly on CSeg, SYSU-CD, PSCD, VL-CMU-CD, and their unaligned variants achieves performance competitive with or superior to dataset specific models, highlighting text conditioning as a powerful approach for generalizable SCD. Our code and dataset are available at https://joshuakgao.github.io/viewdelta/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。