在线学习新场景的3D重建框架，解决传统方法一致性差问题。
Online3R: Online Learning for Consistent Sequential Reconstruction Based on Geometry Foundation Model
- 用可学习的轻量视觉提示微调冻结的几何基础模型，适应新环境。
- 通过局部-全局自监督学习，在无真值情况下实现高效在线更新。
- 适合需要持续适应新场景的实时3D重建应用，如机器人导航。
我们提出Online3R，一种基于几何基础模型的在线序列重建框架，可通过在线学习适应新场景，有效解决重建不一致问题。具体地，我们在预训练且冻结的几何基础模型中引入一组可学习的轻量级视觉提示，以捕捉新环境的知识，同时保留基础模型的几何预测能力。为解决测试时缺乏真值和更新效率低的问题，我们设计了一种局部-全局自监督学习策略：局部一致性约束作用于中间结果与先前局部融合结果，生成高质量伪真值信号；全局一致性约束则施加于长距离稀疏关键帧而非每帧，使模型能高效学习长轨迹上的稳定预测。实验表明，Online3R在多个基准上超越现有最先进方法。
原文摘要 · Abstract (English)
We present Online3R, a new sequential reconstruction framework that is capable of adapting to new scenes through online learning, effectively resolving inconsistency issues. Specifically, we introduce a set of learnable lightweight visual prompts into a pretrained, frozen geometry foundation model to capture the knowledge of new environments while preserving the fundamental capability of the foundation model for geometry prediction. To solve the problems of missing groundtruth and the requirement of high efficiency when updating these visual prompts at test time, we introduce a local-global self-supervised learning strategy by enforcing the local and global consistency constraints on predictions. The local consistency constraints are conducted on intermediate and previously local fused results, enabling the model to be trained with high-quality pseudo groundtruth signals; the global consistency constraints are operated on sparse keyframes spanning long distances rather than per frame, allowing the model to learn from a consistent prediction over a long trajectory in an efficient way. Our experiments demonstrate that Online3R outperforms previous state-of-the-art methods on various benchmarks. Project page: https://shunkaizhou.github.io/online3r-1.0/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。