用3D高斯+运动轨迹场实现单目视频动态场景重建
3D Gaussian Representations with Motion Trajectory Field for Dynamic Scene Reconstruction
- 将3D高斯与运动轨迹场结合，分离动态物体与静态背景
- 在自定义数据集上达到最佳视图合成与运动轨迹恢复效果
- 适合机器人视觉、AR/VR等需动态场景建模的场景
本文针对单目视频中动态场景的新视角合成与运动重建难题提出新方法，该问题对机器人应用至关重要。尽管神经辐射场（NeRF）和3D高斯泼溅（3DGS）在静态场景渲染上表现卓越，但扩展至动态场景仍具挑战。本文提出将3DGS与运动轨迹场相结合的新方法，可精确处理复杂物体运动并生成物理合理的运动轨迹。通过解耦动态物体与静态背景，方法以紧凑方式优化运动轨迹场，引入时不变运动系数与共享运动轨迹基函数，捕捉复杂运动模式同时降低优化复杂度。大量实验表明，该方法在单目视频的新视角合成与运动轨迹恢复上均达到当前最优性能，显著提升动态场景重建能力。
原文摘要 · Abstract (English)
This paper addresses the challenge of novel-view synthesis and motion reconstruction of dynamic scenes from monocular video, which is critical for many robotic applications. Although Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) have demonstrated remarkable success in rendering static scenes, extending them to reconstruct dynamic scenes remains challenging. In this work, we introduce a novel approach that combines 3DGS with a motion trajectory field, enabling precise handling of complex object motions and achieving physically plausible motion trajectories. By decoupling dynamic objects from static background, our method compactly optimizes the motion trajectory field. The approach incorporates time-invariant motion coefficients and shared motion trajectory bases to capture intricate motion patterns while minimizing optimization complexity. Extensive experiments demonstrate that our approach achieves state-of-the-art results in both novel-view synthesis and motion trajectory recovery from monocular video, advancing the capabilities of dynamic scene reconstruction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。