提出实时4D动态场景重建框架，提升自动驾驶感知精度与速度。
Driv3R: Learning Dense 4D Reconstruction for Autonomous Driving
- 直接从多视角图像回归帧级点云，避免复杂对齐过程。
- 在nuScenes上实现15倍更快推理，动态区域重建更精准。
- 适合自动驾驶中需快速高精度3D感知的工程应用。
实时动态场景的4D重建仍是自动驾驶感知的核心挑战。现有方法多依赖自监督深度估计或多模态传感器融合。本文提出基于DUSt3R的Driv3R框架，直接从多视角图像序列回归每帧点云图。为实现流式稠密重建，我们维护一个记忆池，用于推理跨传感器空间关系与动态时间上下文，以增强多视角3D一致性与时间整合效果。此外，引入4D光流预测器识别场景中的运动物体，引导网络重点重建动态区域。最后，通过无优化方式将各帧点云一致对齐至世界坐标系。我们在大规模nuScenes数据集上进行大量实验，结果表明Driv3R在4D动态场景重建上优于先前方法，推理速度比需全局对齐的方法快15倍。代码已开源。
原文摘要 · Abstract (English)
Realtime 4D reconstruction for dynamic scenes remains a crucial challenge for autonomous driving perception. Most existing methods rely on depth estimation through self-supervision or multi-modality sensor fusion. In this paper, we propose Driv3R, a DUSt3R-based framework that directly regresses per-frame point maps from multi-view image sequences. To achieve streaming dense reconstruction, we maintain a memory pool to reason both spatial relationships across sensors and dynamic temporal contexts to enhance multi-view 3D consistency and temporal integration. Furthermore, we employ a 4D flow predictor to identify moving objects within the scene to direct our network focus more on reconstructing these dynamic regions. Finally, we align all per-frame pointmaps consistently to the world coordinate system in an optimization-free manner. We conduct extensive experiments on the large-scale nuScenes dataset to evaluate the effectiveness of our method. Driv3R outperforms previous frameworks in 4D dynamic scene reconstruction, achieving 15x faster inference speed compared to methods requiring global alignment. Code: https://github.com/Barrybarry-Smith/Driv3R.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。