arXiv:2603.20778cs.CV2026-03被引 5

用视频直接对齐3D地图,实现无人机精准定位与目标追踪。

PiLoT: Neural Pixel-to-3D Registration for UAV-based Ego and Target Geo-localization

  • 视频流与地理参考3D地图直接神经注册,取代传统传感器融合。
  • 在Jetson Orin上实现实时运行(超25 FPS),零样本泛化至真实场景。
  • 适合需要低成本、高鲁棒性定位的无人机应用,尤其无卫星信号环境。

我们提出PiLoT,一种统一框架,用于无人机自位与目标地理定位。传统方法依赖解耦流程:用GNSS与视觉惯性里程计(VIO)估计自身姿态,用激光测距仪等主动传感器定位目标。这些方法在无GNSS环境下易失效,且硬件成本高、复杂度大。PiLoT打破这一范式,直接将实时视频流与地理参考3D地图进行神经注册。为实现鲁棒、准确、实时性能,提出三项关键贡献:1)双线程引擎,解耦地图渲染与核心定位线程,兼顾低延迟与无漂移精度;2)大规模合成数据集,含精确几何标注(相机位姿、深度图),可训练轻量网络,实现从仿真到真实数据的零样本泛化;3)联合神经引导随机梯度优化器(JNGO),即使在剧烈运动下也能实现稳健收敛。在多个公开及新收集基准上的评估表明,PiLoT优于现有最先进方法,且在NVIDIA Jetson Orin平台实现超过25 FPS的运行速度。代码与数据集已开源。

原文摘要 · Abstract (English)

We present PiLoT, a unified framework that tackles UAV-based ego and target geo-localization. Conventional approaches rely on decoupled pipelines that fuse GNSS and Visual-Inertial Odometry (VIO) for ego-pose estimation, and active sensors like laser rangefinders for target localization. However, these methods are susceptible to failure in GNSS-denied environments and incur substantial hardware costs and complexity. PiLoT breaks this paradigm by directly registering live video stream against a geo-referenced 3D map. To achieve robust, accurate, and real-time performance, we introduce three key contributions: 1) a Dual-Thread Engine that decouples map rendering from core localization thread, ensuring both low latency while maintaining drift-free accuracy; 2) a large-scale synthetic dataset with precise geometric annotations (camera pose, depth maps). This dataset enables the training of a lightweight network that generalizes in a zero-shot manner from simulation to real data; and 3) a Joint Neural-Guided Stochastic-Gradient Optimizer (JNGO) that achieves robust convergence even under aggressive motion. Evaluations on a comprehensive set of public and newly collected benchmarks show that PiLoT outperforms state-of-the-art methods while running over 25 FPS on NVIDIA Jetson Orin platform. Our code and dataset is available at: https://github.com/Choyaa/PiLoT.

无人机定位神经注册3D地图实时系统

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。