全球城市级视频地理定位，首次实现跨城市、跨洲的精准定位。
CityGuessr: City-Level Video Geo-Localization on a Global Scale
- 基于变压器架构，引入自交叉注意力融合场景特征。
- 在68,269个全球城市视频上达到城市准确率37.4%。
- 适合做跨域视频定位、地理信息提取的研究者参考。
视频地理定位是当前重要问题，仅凭一段视频即可判断拍摄地点，具有广泛应用价值。现有研究多聚焦于图像层面的全球定位，而视频层面的全球定位仍属空白。尽管近年有局部视频定位工作，但均局限于特定区域。为此，本文提出全球尺度的视频地理定位任务，目标是层次化预测视频拍摄的城、省/州、国家和大洲。由于缺乏大规模覆盖全球的视频数据集，本文构建了新数据集CityGuessr68k，包含来自全球166个城市的68,269段视频。同时提出一种新型基线方法，采用基于变压器的架构，设计自交叉注意力模块以融合场景信息，并引入文本标签对齐策略，在特征空间中蒸馏文本标签知识。为进一步提升定位精度，还使用软场景标签。实验在自建数据集及Mapillary MSLS上验证了方法有效性。代码与数据已开源。
原文摘要 · Abstract (English)
Video geolocalization is a crucial problem in current times. Given just a video, ascertaining where it was captured from can have a plethora of advantages. The problem of worldwide geolocalization has been tackled before, but only using the image modality. Its video counterpart remains relatively unexplored. Meanwhile, video geolocalization has also garnered some attention in the recent past, but the existing methods are all restricted to specific regions. This motivates us to explore the problem of video geolocalization at a global scale. Hence, we propose a novel problem of worldwide video geolocalization with the objective of hierarchically predicting the correct city, state/province, country, and continent, given a video. However, no large scale video datasets that have extensive worldwide coverage exist, to train models for solving this problem. To this end, we introduce a new dataset, CityGuessr68k comprising of 68,269 videos from 166 cities all over the world. We also propose a novel baseline approach to this problem, by designing a transformer-based architecture comprising of an elegant Self-Cross Attention module for incorporating scenes as well as a TextLabel Alignment strategy for distilling knowledge from textlabels in feature space. To further enhance our location prediction, we also utilize soft-scene labels. Finally we demonstrate the performance of our method on our new dataset as well as Mapillary(MSLS). Our code and datasets are available at: https://github.com/ParthPK/CityGuessr
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。