用3D高斯泼溅优化相机位姿,无需重训练即可适配新场景。
Camera Pose Refinement via 3D Gaussian Splatting
- 基于3DGS渲染多视角图像,利用对极几何约束迭代优化位姿。
- 在7-Scenes和Cambridge Landmarks上位姿误差降低超40%。
- 兼容多种特征提取器,适用于无额外训练的通用场景。
相机位姿精修旨在提升3D计算机视觉应用中初始位姿估计的精度。现有方法多依赖特定描述符或专用网络的2D-3D对应关系,需为不同描述符重建场景或为每场景重新训练网络。部分最新方法通过特征相似性推断位姿,但缺乏几何约束导致精度不足。为此,我们提出基于3D高斯泼溅(3DGS)的新型精修框架GS-SMC。得益于3DGS的广泛应用,该方法可直接使用已有3DGS模型渲染新视角,实现无需额外训练或微调的轻量级方案。具体而言,我们引入一种迭代优化方法,利用查询图像与多个渲染图像间的对极几何约束精修相机位姿,并可灵活选择特征提取器与匹配器建立约束。在7-Scenes与Cambridge Landmarks数据集上的大量实验表明,本方法优于当前最优精修方法,在7-Scenes上实现53.3%与56.9%的中位平移与旋转误差降低,在Cambridge上分别达40.7%与53.2%。
原文摘要 · Abstract (English)
Camera pose refinement aims at improving the accuracy of initial pose estimation for applications in 3D computer vision. Most refinement approaches rely on 2D-3D correspondences with specific descriptors or dedicated networks, requiring reconstructing the scene again for a different descriptor or fully retraining the network for each scene. Some recent methods instead infer pose from feature similarity, but their lack of geometry constraints results in less accuracy. To overcome these limitations, we propose a novel camera pose refinement framework leveraging 3D Gaussian Splatting (3DGS), referred to as GS-SMC. Given the widespread usage of 3DGS, our method can employ an existing 3DGS model to render novel views, providing a lightweight solution that can be directly applied to diverse scenes without additional training or fine-tuning. Specifically, we introduce an iterative optimization approach, which refines the camera pose using epipolar geometric constraints among the query and multiple rendered images. Our method allows flexibly choosing feature extractors and matchers to establish these constraints. Extensive empirical evaluations on the 7-Scenes and the Cambridge Landmarks datasets demonstrate that our method outperforms state-of-the-art camera pose refinement approaches, achieving 53.3% and 56.9% reductions in median translation and rotation errors on 7-Scenes, and 40.7% and 53.2% on Cambridge.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。