arXiv:2504.19819cs.CV2025-04CVPR

无需初始位姿,通过连续运动建模实现单目视频的高精度3D重建。

Joint Optimization of Neural Radiance Fields and Continuous Camera Motion from a Monocular Video

  • 将相机运动建模为随时间变化的速度与角速度,通过相对运动积分求解位姿。
  • 在Co3D和Scannet数据集上,位姿和深度估计优于现有方法,新视角合成性能相当。
  • 适合无初始位姿、大旋转等挑战场景,尤其适用于单目视频三维重建。

神经辐射场(NeRF)虽能精准表示三维几何结构,但需训练前精确已知相机位姿。现有联合优化方法依赖良好的初始位姿或深度先验,在大幅旋转等复杂场景下表现不佳,因将每帧相机映射到全局坐标系。本文提出新方法,通过将连续相机运动建模为时变角速度与线速度,先学习相机间的相对运动,再通过积分获得整体位姿,仅需在视频中某一时刻定义世界坐标系。该方法基于时变NeRF,从相邻帧中学习局部场景几何与运动信息。所学运动可用于精调NeRF,实现完整场景重建。在Co3D与Scannet数据集上的实验表明,本方法在相机位姿与深度估计方面表现更优,新视角合成效果接近当前最优水平。

原文摘要 · Abstract (English)

Neural Radiance Fields (NeRF) has demonstrated its superior capability to represent 3D geometry but require accurately precomputed camera poses during training. To mitigate this requirement, existing methods jointly optimize camera poses and NeRF often relying on good pose initialisation or depth priors. However, these approaches struggle in challenging scenarios, such as large rotations, as they map each camera to a world coordinate system. We propose a novel method that eliminates prior dependencies by modeling continuous camera motions as time-dependent angular velocity and velocity. Relative motions between cameras are learned first via velocity integration, while camera poses can be obtained by aggregating such relative motions up to a world coordinate system defined at a single time step within the video. Specifically, accurate continuous camera movements are learned through a time-dependent NeRF, which captures local scene geometry and motion by training from neighboring frames for each time step. The learned motions enable fine-tuning the NeRF to represent the full scene geometry. Experiments on Co3D and Scannet show our approach achieves superior camera pose and depth estimation and comparable novel-view synthesis performance compared to state-of-the-art methods. Our code is available at https://github.com/HoangChuongNguyen/cope-nerf.

NeRF单目重建运动建模三维重建

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。