用3D高斯点云生成图像，快速精准定位相机姿态。
GSFeatLoc: Visual Localization Using Feature Correspondence on 3D Gaussian Splatting
- 通过3DGS生成合成图像，建立2D-2D对应关系。
- 利用深度图提升为2D-3D对应，解PnP得最终位姿。
- 定位快至0.1秒，容忍初始误差55°旋转、1.1单位平移。
本文提出一种基于3D高斯点云（3DGS）场景表示的视觉定位方法。首先，使用3DGS在初始位姿估计下渲染合成RGBD图像；其次，建立查询图像与合成图像间的2D-2D对应关系；第三，利用深度图将2D-2D对应提升为2D-3D对应，并求解透视n点（PnP）问题获得最终位姿估计。在三个现有数据集共38个场景、超过2700张测试图像上的评估显示，该方法相比基于光度损失优化的基线方法，推理时间降低两个数量级以上（从10秒以上降至0.1秒以内），且估计误差显著降低。结果还表明，该方法对初始位姿误差具有强鲁棒性，可容忍高达55°的旋转误差和1.1单位的平移误差（以场景尺度归一化），在Synthetic NeRF和Mip-NeRF360数据集上90%图像的最终误差低于5°旋转和0.05单位平移，在更挑战的Tanks and Temples数据集上也达到42%的成功率。
原文摘要 · Abstract (English)
In this paper, we present a method for localizing a query image with respect to a precomputed 3D Gaussian Splatting (3DGS) scene representation. First, the method uses 3DGS to render a synthetic RGBD image at some initial pose estimate. Second, it establishes 2D-2D correspondences between the query image and this synthetic image. Third, it uses the depth map to lift the 2D-2D correspondences to 2D-3D correspondences and solves a perspective-n-point (PnP) problem to produce a final pose estimate. Results from evaluation across three existing datasets with 38 scenes and over 2,700 test images show that our method significantly reduces both inference time (by over two orders of magnitude, from more than 10 seconds to as fast as 0.1 seconds) and estimation error compared to baseline methods that use photometric loss minimization. Results also show that our method tolerates large errors in the initial pose estimate of up to 55° in rotation and 1.1 units in translation (normalized by scene scale), achieving final pose errors of less than 5° in rotation and 0.05 units in translation on 90% of images from the Synthetic NeRF and Mip-NeRF360 datasets and on 42% of images from the more challenging Tanks and Temples dataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。