通过几何一致性正则化,让快速相机位姿回归逼近高精度对应方法。
GeLoc3r: Enhancing Relative Camera Pose Regression with Geometric Consistency Regularization
- 训练时用真值深度生成3D-2D对应点,通过加权RANSAC构建几何一致性损失。
- 在CO3Dv2上达到40.45%的AUC@5°,相比原模型提升16%相对性能。
- 保持25ms推理速度,适合需要高速高精度位姿估计的应用场景。
Prior ReLoc3R 在25ms内实现快速推理并达到领先回归精度,但分析发现其内部表征存在细微几何不一致,限制了其向基于对应的方法(如MASt3R,每对需300ms)精度上限的突破。本文提出GeLoc3r,一种通过几何一致性正则化(GCR)增强相对相机位姿估计的新方法。训练阶段,利用真值深度生成密集3D-2D对应点,由FusionTransformer学习对应点重要性,并通过加权RANSAC计算几何一致位姿,构建一致性损失以传递几何知识至回归网络。与FAR方法需在推理时同时进行回归与几何求解不同,GeLoc3r仅使用优化后的回归头,保持25ms推理速度,接近MASt3R的高精度。在多个挑战性基准测试中,GeLoc3r持续优于ReLoc3R:CO3Dv2上达40.45% vs. 34.85% AUC@5°(相对提升16%),RealEstate10K上68.66% vs. 66.70%,MegaDepth1500上50.45% vs. 49.60%。通过训练时注入几何一致性而非推理时强制执行,GeLoc3r实现了神经网络学习相机几何的范式转变,兼具回归方法的速度与对应方法的几何理解能力。
原文摘要 · Abstract (English)
Prior ReLoc3R achieves breakthrough performance with fast 25ms inference and state-of-the-art regression accuracy, yet our analysis reveals subtle geometric inconsistencies in its internal representations that prevent reaching the precision ceiling of correspondence-based methods like MASt3R (which require 300ms per pair). In this work, we present GeLoc3r, a novel approach to relative camera pose estimation that enhances pose regression methods through Geometric Consistency Regularization (GCR). GeLoc3r overcomes the speed-accuracy dilemma by training regression networks to produce geometrically consistent poses without inference-time geometric computation. During training, GeLoc3r leverages ground-truth depth to generate dense 3D-2D correspondences, weights them using a FusionTransformer that learns correspondence importance, and computes geometrically-consistent poses via weighted RANSAC. This creates a consistency loss that transfers geometric knowledge into the regression network. Unlike FAR method which requires both regression and geometric solving at inference, GeLoc3r only uses the enhanced regression head at test time, maintaining ReLoc3R's fast speed and approaching MASt3R's high accuracy. On challenging benchmarks, GeLoc3r consistently outperforms ReLoc3R, achieving significant improvements including 40.45% vs. 34.85% AUC@5° on the CO3Dv2 dataset (16% relative improvement), 68.66% vs. 66.70% AUC@5° on RealEstate10K, and 50.45% vs. 49.60% on MegaDepth1500. By teaching geometric consistency during training rather than enforcing it at inference, GeLoc3r represents a paradigm shift in how neural networks learn camera geometry, achieving both the speed of regression and the geometric understanding of correspondence methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。