用Transformer直接回归相对位姿,实现快速高精度单目里程计
FVO: Fast Visual Odometry with Transformers
- 将单目里程计建模为直接相对位姿回归问题,摒弃传统后处理步骤
- 在多个基准上达到竞品水平或更好性能,推理速度接近最快基线的2倍
- 引入置信度感知融合机制,提升轨迹估计鲁棒性,适合实时系统部署
结合深度学习与经典优化的混合方法已成为视觉里程计(VO)的主流。这类模型通过神经网络预测与光束法平差结合,实现高精度相机轨迹估计。然而,混合方法在速度和能力上仍不及纯端到端方案。现有框架依赖大规模预训练3D网络预测几何,这些后端因训练时保持尺度模糊且冻结,导致整个流程无法估计绝对尺度;同时缓慢的优化与后处理严重拖慢推理速度。本文提出将单目视觉里程计直接建模为相对位姿回归问题,彻底移除后处理环节。该方法训练一个高速、高容量的Transformer,仅使用相机位姿作为监督信号,直接预测相对位姿及对应置信度。更重要的是,采用置信度感知的推理策略,聚合重叠的位姿预测以提升轨迹估计鲁棒性。在多个视觉里程计基准测试中,所提方法FVO成功利用多样化数据,实现竞争性或更优性能,且推理速度接近当前最快基线的2倍。
原文摘要 · Abstract (English)
Hybrid pipelines that combine deep learning with classical optimization have established themselves as the dominant approach to visual odometry (VO). By integrating neural network predictions with bundle adjustment, these models estimate camera trajectories with high accuracy. Still, hybrid VO methods fall short of the speed and capabilities of pure end-to-end approaches. Current hybrid frameworks rely on massive, pre-trained 3D networks to predict geometry. Because these backends are trained to be scale-ambiguous and frozen rather than retrained, the pipelines essentially inherit this limitation and, by design, fails to estimate absolute scale. Furthermore, their slow optimization and post-processing steps bottleneck the pipeline's inference speed. We propose to replace post-processing entirely by formulating monocular visual odometry as a direct relative pose regression problem. This formulation enables us to train a fast, high-capacity transformer to predict relative camera poses and corresponding confidences using only camera poses as supervision. More importantly, it allows us to employ a confidence-aware inference scheme that aggregates overlapping pose predictions for robust trajectory estimation. We demonstrate on multiple visual odometry benchmarks that our method, Fast Visual Odometry (FVO), successfully leverages diverse data to achieve competitive or superior performance while being nearly 2 times faster than the fastest baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。