arXiv:2604.08542cs.CV2026-04被引 21

通过测试时训练实现大规模3D重建,提升长期视频的精度与一致性。

Scal3R: Scalable Test-Time Training for Large-Scale 3D Reconstruction

论文配图:Scal3R: Scalable Test-Time Training for Large-Scale 3D Reconstruction
图 1 · 摘自论文原文
  • 引入轻量神经全局上下文表征,测试时自监督快速适应。
  • 在KITTI和Oxford Spires上达到领先位姿精度与最优3D重建效果。
  • 适合需要长序列稳定重建的应用,如自动驾驶、数字孪生。

本文针对从长视频序列中进行大规模3D场景重建的任务。近期的前馈重建模型通过直接从RGB图像回归3D几何结构,在无需显式3D先验或几何约束的情况下取得良好效果。然而,这些方法在长时间序列下常因内存容量有限、难以有效捕捉全局上下文线索而导致重建精度与一致性下降。相比之下,人类能自然利用对场景的整体理解来指导局部感知。受此启发,我们提出一种新型神经全局上下文表征,可高效压缩并保留远距离场景信息,使模型能够利用广泛上下文线索提升重建精度与一致性。该上下文表示通过一组轻量级神经子网络实现,可在测试时通过自监督目标快速适应,显著增加内存容量且计算开销可控。在KITTI Odometry和Oxford Spires等多个大规模基准上的实验表明,该方法能有效处理超大规模场景,在保持高效的同时实现领先的位姿精度与最先进的3D重建精度。代码已开源:https://zju3dv.github.io/scal3r。

原文摘要 · Abstract (English)

This paper addresses the task of large-scale 3D scene reconstruction from long video sequences. Recent feed-forward reconstruction models have shown promising results by directly regressing 3D geometry from RGB images without explicit 3D priors or geometric constraints. However, these methods often struggle to maintain reconstruction accuracy and consistency over long sequences due to limited memory capacity and the inability to effectively capture global contextual cues. In contrast, humans can naturally exploit the global understanding of the scene to inform local perception. Motivated by this, we propose a novel neural global context representation that efficiently compresses and retains long-range scene information, enabling the model to leverage extensive contextual cues for enhanced reconstruction accuracy and consistency. The context representation is realized through a set of lightweight neural sub-networks that are rapidly adapted during test time via self-supervised objectives, which substantially increases memory capacity without incurring significant computational overhead. The experiments on multiple large-scale benchmarks, including the KITTI Odometry~\cite{Geiger2012CVPR} and Oxford Spires~\cite{tao2025spires} datasets, demonstrate the effectiveness of our approach in handling ultra-large scenes, achieving leading pose accuracy and state-of-the-art 3D reconstruction accuracy while maintaining efficiency. Code is available at https://zju3dv.github.io/scal3r.

3D重建测试时训练大规模场景上下文建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。