用前馈网络加速RGB SLAM,速度提升90%以上
Pseudo Depth Meets Gaussian: A Feed-forward RGB SLAM Baseline
- 用3D高斯表示+前馈递归模块预测位姿,替代传统优化
- 在Replica和TUM-RGBD上达到SplaTAM水平,跟踪提速超90%
- 适合需要实时重建的机器人或AR应用
从无位姿约束的RGB流中逐步恢复真实尺度三维几何是一项挑战性任务,要求对输入数据假设最少。现有方法可分为端到端和基于视觉SLAM两类,均存在长序列处理困难或依赖缓慢的测试时优化、深度传感器等问题。本文首次将深度估计算法集成至RGB-D SLAM系统,但发现预测深度存在几何细节不准的问题。进一步研究发现,3D高斯映射可有效解决此问题。基于此,提出一种基于3D高斯的在线三维重建方法,结合前馈递归预测模块,直接从光流推断相机位姿,取代低效的测试时优化,显著提升追踪速度。此外,引入局部图渲染技术以增强前馈位姿预测的鲁棒性。在Replica与TUM-RGBD数据集上的实验结果及真实场景部署演示表明,本方法性能媲美当前最优的SplaTAM,同时追踪时间减少超过90%。
原文摘要 · Abstract (English)
Incrementally recovering real-sized 3D geometry from a pose-free RGB stream is a challenging task in 3D reconstruction, requiring minimal assumptions on input data. Existing methods can be broadly categorized into end-to-end and visual SLAM-based approaches, both of which either struggle with long sequences or depend on slow test-time optimization and depth sensors. To address this, we first integrate a depth estimator into an RGB-D SLAM system, but this approach is hindered by inaccurate geometric details in predicted depth. Through further investigation, we find that 3D Gaussian mapping can effectively solve this problem. Building on this, we propose an online 3D reconstruction method using 3D Gaussian-based SLAM, combined with a feed-forward recurrent prediction module to directly infer camera pose from optical flow. This approach replaces slow test-time optimization with fast network inference, significantly improving tracking speed. Additionally, we introduce a local graph rendering technique to enhance robustness in feed-forward pose prediction. Experimental results on the Replica and TUM-RGBD datasets, along with a real-world deployment demonstration, show that our method achieves performance on par with the state-of-the-art SplaTAM, while reducing tracking time by more than 90\%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。