解决3D视觉模型在线重建中的时空不一致性问题
TALO: Pushing 3D Vision Foundation Models Towards Globally Consistent Online Reconstruction
- 用薄板样条实现高自由度长期对齐,通过全局控制点修正空间偏差
- 在多数据集和相机配置下,轨迹误差降低,几何一致性显著提升
- 无需修改主模型,兼容单目与环视相机,适合自动驾驶等实时场景
3D视觉基础模型通过单次前向传播即可从未标定图像中重建关键3D属性,展现出强大泛化能力。然而,在驾驶等在线场景中,预测基于时间窗口进行,维持时序一致性极具挑战。现有方法通过求解全局变换对齐连续预测,但分析表明其存在假设有效性不足、局部对齐范围有限及噪声几何下鲁棒性差等根本缺陷。本文提出一种基于薄板样条的高自由度长期对齐框架,利用全局传播的控制点校正空间变化的不一致。同时采用无点子图注册设计,天然具备对噪声几何预测的鲁棒性。该框架完全即插即用,兼容多种3D基础模型与相机配置(如单目或环视)。大量实验表明,本方法在多个数据集、主干模型和相机设置下均持续生成更连贯的几何结构并降低轨迹误差,凸显其鲁棒性与通用性。代码已开源。
原文摘要 · Abstract (English)
3D vision foundation models have shown strong generalization in reconstructing key 3D attributes from uncalibrated images through a single feed-forward pass. However, when deployed in online settings such as driving scenarios, predictions are made over temporal windows, making it non-trivial to maintain consistency across time. Recent strategies align consecutive predictions by solving global transformation, yet our analysis reveals their fundamental limitations in assumption validity, local alignment scope, and robustness under noisy geometry. In this work, we propose a higher-DOF and long-term alignment framework based on Thin Plate Spline, leveraging globally propagated control points to correct spatially varying inconsistencies. In addition, we adopt a point-agnostic submap registration design that is inherently robust to noisy geometry predictions. The proposed framework is fully plug-and-play, compatible with diverse 3D foundation models and camera configurations (e.g., monocular or surround-view). Extensive experiments demonstrate that our method consistently yields more coherent geometry and lower trajectory errors across multiple datasets, backbone models, and camera setups, highlighting its robustness and generality. Code is available at https://github.com/Xian-Bei/TALO.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。