用视频帧精准定位全球地理位置，比现有方法快25%且更准。
VidTAG: Temporally Aligned Video to GPS Geolocalization with Denoising Sequence Prediction at a Global Scale
- 双编码器框架结合自监督与语言特征，实现帧到地理坐标的匹配。
- 在1公里阈值下相比GeoCLIP提升20%，城市级定位准确率提高25%。
- 适合需要全球范围视频定位的司法取证、社交媒体分析等场景。
视频地理定位旨在确定视频拍摄的精确GPS坐标并还原其轨迹，广泛应用于司法取证、社交媒体分析及探索领域。现有分类方法仅能粗粒度定位至城市级别，难以捕捉细节；图像检索方法因需大规模图像库而无法在全局应用。相比之下，构建地理坐标库成本低、易实现。本文提出VidTAG，一种双编码器框架，利用自监督与语言对齐特征进行帧到GPS的检索。为解决视频预测的时间不一致性，引入TempGeo模块对齐帧嵌入，并设计GeoRefiner模块，通过编码器-解码器结构基于对齐后的帧嵌入优化GPS特征。在Mapillary（MSLS）和GAMa数据集上的评估表明，该模型可生成时间一致的轨迹，在1公里阈值下相较GeoCLIP提升20%；在全球粗粒度视频地理定位任务（CityGuessr68k）中优于当前最优方法25%。本方法实现了细粒度视频地理定位，为后续研究奠定坚实基础。
原文摘要 · Abstract (English)
The task of video geolocalization aims to determine the precise GPS coordinates of a video's origin and map its trajectory; with applications in forensics, social media, and exploration. Existing classification-based approaches operate at a coarse city-level granularity and fail to capture fine-grained details, while image retrieval methods are impractical on a global scale due to the need for extensive image galleries which are infeasible to compile. Comparatively, constructing a gallery of GPS coordinates is straightforward and inexpensive. We propose VidTAG, a dual-encoder framework that performs frame-to-GPS retrieval using both self-supervised and language-aligned features. To address temporal inconsistencies in video predictions, we introduce the TempGeo module, which aligns frame embeddings, and the GeoRefiner module, an encoder-decoder architecture that refines GPS features using the aligned frame embeddings. Evaluations on Mapillary (MSLS) and GAMa datasets demonstrate our model's ability to generate temporally consistent trajectories and outperform baselines, achieving a 20% improvement at the 1 km threshold over GeoCLIP. We also beat current State-of-the-Art by 25% on global coarse grained video geolocalization (CityGuessr68k). Our approach enables fine-grained video geolocalization and lays a strong foundation for future research. More details on the project webpage: https://parthpk.github.io/vidtag_webpage/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。