arXiv:2503.16318cs.CV2025-03ICCV被引 46

动态点图让3D重建能处理运动场景,统一解决时序几何任务。

Dynamic Point Maps: A Versatile Representation for Dynamic 3D Reconstruction

  • 引入时空点图,将动态3D任务映射为可学习的点云预测
  • 在视频深度、场景流等4项任务上达最新性能,优于现有方法
  • 适用于需要追踪物体运动或重建动态场景的研究者

DUSt3R表明,多视角几何中的相机参数估计、3D重建和图像对应等问题,可统一为预测一对视角不变的点图(即像素对齐的点云)。该方法简洁有力,但无法处理动态场景。为此,本文提出动态点图(DPM),将标准点图扩展至4维,支持运动分割、场景流估计、3D物体追踪与2D对应等任务。核心思想是引入时间维度后存在多种时空参考系,我们识别出一组最小组合,可通过网络回归以解决上述子任务。模型在合成与真实数据混合训练,并在视频深度预测、动态点云重建、3D场景流及物体姿态追踪等多个基准上评估,均达到当前最优表现。代码、模型与补充结果见https://www.robots.ox.ac.uk/~vgg/research/dynamic-point-maps/。

原文摘要 · Abstract (English)

DUSt3R has recently shown that one can reduce many tasks in multi-view geometry, including estimating camera intrinsics and extrinsics, reconstructing the scene in 3D, and establishing image correspondences, to the prediction of a pair of viewpoint-invariant point maps, i.e., pixel-aligned point clouds defined in a common reference frame. This formulation is elegant and powerful, but unable to tackle dynamic scenes. To address this challenge, we introduce the concept of Dynamic Point Maps (DPM), extending standard point maps to support 4D tasks such as motion segmentation, scene flow estimation, 3D object tracking, and 2D correspondence. Our key intuition is that, when time is introduced, there are several possible spatial and time references that can be used to define the point maps. We identify a minimal subset of such combinations that can be regressed by a network to solve the sub tasks mentioned above. We train a DPM predictor on a mixture of synthetic and real data and evaluate it across diverse benchmarks for video depth prediction, dynamic point cloud reconstruction, 3D scene flow and object pose tracking, achieving state-of-the-art performance. Code, models and additional results are available at https://www.robots.ox.ac.uk/~vgg/research/dynamic-point-maps/.

3D重建动态场景点云多视角几何

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。