arXiv:2510.14960cs.CVcs.AI2025-10ICCV被引 8

用时空对应关系将3D重建扩展为动态4D,实现完整运动场景恢复。

C4D: 4D Made from 3D through Dual Correspondences

  • 引入短时光流与长时点跟踪双对应关系
  • 通过动态感知追踪器分离动/静物体,提升重建精度
  • 适用于需要精确运动建模的视觉任务,如相机位姿估计

从单目视频中恢复4D(动态几何与相机位姿)是极具挑战性的问题。尽管基于点图的3D重建方法(如DUSt3R)在静态场景中取得显著进展,直接应用于动态场景会导致结果失真,因运动物体违反多视角几何约束。为此,我们提出C4D框架,通过时间对应关系将现有3D重建范式拓展至4D。C4D不仅预测点图,还捕捉短时光流与长时点跟踪两类对应关系。训练了一个动态感知点追踪器,提供额外运动信息,辅助生成运动掩码以分离运动物体与静态背景,从而为动态场景提供更可靠的引导。同时,引入一组动态场景优化目标,实现每帧3D几何与相机参数的恢复。对应关系还将2D轨迹升维为平滑3D轨迹,实现完整的4D重建。实验表明,该框架可实现完整的4D恢复,并在深度估计、相机位姿估计和点跟踪等下游任务中表现优异。

原文摘要 · Abstract (English)

Recovering 4D from monocular video, which jointly estimates dynamic geometry and camera poses, is an inevitably challenging problem. While recent pointmap-based 3D reconstruction methods (e.g., DUSt3R) have made great progress in reconstructing static scenes, directly applying them to dynamic scenes leads to inaccurate results. This discrepancy arises because moving objects violate multi-view geometric constraints, disrupting the reconstruction. To address this, we introduce C4D, a framework that leverages temporal Correspondences to extend existing 3D reconstruction formulation to 4D. Specifically, apart from predicting pointmaps, C4D captures two types of correspondences: short-term optical flow and long-term point tracking. We train a dynamic-aware point tracker that provides additional mobility information, facilitating the estimation of motion masks to separate moving elements from the static background, thus offering more reliable guidance for dynamic scenes. Furthermore, we introduce a set of dynamic scene optimization objectives to recover per-frame 3D geometry and camera parameters. Simultaneously, the correspondences lift 2D trajectories into smooth 3D trajectories, enabling fully integrated 4D reconstruction. Experiments show that our framework achieves complete 4D recovery and demonstrates strong performance across multiple downstream tasks, including depth estimation, camera pose estimation, and point tracking. Project Page: https://littlepure2333.github.io/C4D

4D重建动态场景点追踪单目视频

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。