arXiv:2411.13610cs.CV2024-11ICCV被引 23

将无人机视频转为俯视图,提升地理定位精度与鲁棒性

Video2BEV: Transforming Drone Videos to BEVs for Video-based Geo-localization

  • 用高斯点阵重建3D场景,生成无畸变的俯视图
  • 在新数据集UniV上达到领先召回率,低空遮挡下仍稳定
  • 适合做无人机视频定位、多平台匹配的研究者

现有无人机视觉地理定位多采用单张图像匹配,未能充分利用视频特性且易受遮挡和视角差异影响。本文提出视频驱动的地理定位新范式Video2BEV,将无人机视频转换为鸟瞰图(BEV),简化跨平台匹配。通过高斯点阵重建3D场景并生成俯视投影,相比极坐标变换等方法保留更多细节且失真更小。为增强同平台表示学习,引入基于扩散模型的难例生成模块。为验证方法,构建新数据集UniV,扩展自University-1652,包含30°和45°俯仰角飞行路径,帧率最高达10 FPS。在UniV上的实验表明,Video2BEV取得竞争性召回率,尤其在低空、高遮挡场景下表现更优,优于现有视频方法。

原文摘要 · Abstract (English)

Existing approaches to drone visual geo-localization predominantly adopt the image-based setting, where a single drone-view snapshot is matched with images from other platforms. Such task formulation, however, underutilizes the inherent video output of the drone and is sensitive to occlusions and viewpoint disparity. To address these limitations, we formulate a new video-based drone geo-localization task and propose the Video2BEV paradigm. This paradigm transforms the video into a Bird's Eye View (BEV), simplifying the subsequent \textbf{inter-platform} matching process. In particular, we employ Gaussian Splatting to reconstruct a 3D scene and obtain the BEV projection. Different from the existing transform methods, \eg, polar transform, our BEVs preserve more fine-grained details without significant distortion. To facilitate the discriminative \textbf{intra-platform} representation learning, our Video2BEV paradigm also incorporates a diffusion-based module for generating hard negative samples. To validate our approach, we introduce UniV, a new video-based geo-localization dataset that extends the image-based University-1652 dataset. UniV features flight paths at $30^\circ$ and $45^\circ$ elevation angles with increased frame rates of up to 10 frames per second (FPS). Extensive experiments on the UniV dataset show that our Video2BEV paradigm achieves competitive recall rates and outperforms conventional video-based methods. Compared to other competitive methods, our proposed approach exhibits robustness at lower elevations with more occlusions.

视频定位鸟瞰图无人机几何重建

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。