用自监督方法在百万级街景图上检测城市变化，发现小至街区改造的大到房价关联的不平等。
EMPLACE: Self-Supervised Urban Scene Change Detection
- 用自适应三元组损失训练Vision Transformer，无需人工标注
- 在110万张图像数据集上超越现有最佳方法，支持零样本检测
- 可发现城市微小变化并揭示其与房价不平等的关联，适合城市规划者
城市变迁持续影响社区感知与居民生活。城市街景变化检测（USCD）利用计算机视觉捕捉街道变化，有助于更深入理解城市及其居民。传统方法依赖小规模标注数据集的监督学习，限制了在新城市的泛化能力，且需人工定义变化类别。本文提出迄今最大的USCD数据集AC-1M，包含超过110万张图像，并引入自监督方法EMPLACE，基于自适应三元组损失训练视觉变换器。实验表明，EMPLACE在预训练线性微调和零样本设置下均优于当前最优方法。在阿姆斯特丹的案例研究中，该方法成功检测出全城范围内的大小变化，且检测到的变化程度与房价相关，反映出潜在的社会不平等现象。
原文摘要 · Abstract (English)
Urban change is a constant process that influences the perception of neighbourhoods and the lives of the people within them. The field of Urban Scene Change Detection (USCD) aims to capture changes in street scenes using computer vision and can help raise awareness of changes that make it possible to better understand the city and its residents. Traditionally, the field of USCD has used supervised methods with small scale datasets. This constrains methods when applied to new cities, as it requires labour-intensive labeling processes and forces a priori definitions of relevant change. In this paper we introduce AC-1M the largest USCD dataset by far of over 1.1M images, together with EMPLACE, a self-supervising method to train a Vision Transformer using our adaptive triplet loss. We show EMPLACE outperforms SOTA methods both as a pre-training method for linear fine-tuning as well as a zero-shot setting. Lastly, in a case study of Amsterdam, we show that we are able to detect both small and large changes throughout the city and that changes uncovered by EMPLACE, depending on size, correlate with housing prices - which in turn is indicative of inequity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。