用可变形样条实现单目视频实时动态3D高斯建模
SplineGS: Robust Motion-Adaptive Spline for Real-Time Dynamic 3D Gaussians from Monocular Video
- 用少量控制点的样条拟合动态高斯轨迹，支持连续运动建模
- 在单目视频上实现高质量新视角合成，渲染速度提升千倍以上
- 无需预处理，适合真实场景下的动态3D重建
从真实世界的单目视频中合成新视角极具挑战，源于场景动态性及缺乏多视角信息。为此，我们提出SplineGS，一种无需COLMAP的动态3D高斯溅射（3DGS）框架，可在单目视频上实现高质量重建与快速渲染。核心是新型运动自适应样条（MAS），采用少数量控制点的三次埃尔米特样条表示连续动态3D高斯轨迹。针对MAS，我们引入运动自适应控制点剪枝（MACP）方法，在不同运动下建模每个动态高斯的形变，逐步剪枝控制点以保持动态建模完整性。此外，提出相机参数与3D高斯属性联合优化策略，利用光度与几何一致性，消除对结构从运动（SfM）预处理的需求，提升实际场景下的鲁棒性。实验表明，SplineGS在单目视频动态场景的新视角合成质量上显著优于现有方法，渲染速度达数千倍提升。
原文摘要 · Abstract (English)
Synthesizing novel views from in-the-wild monocular videos is challenging due to scene dynamics and the lack of multi-view cues. To address this, we propose SplineGS, a COLMAP-free dynamic 3D Gaussian Splatting (3DGS) framework for high-quality reconstruction and fast rendering from monocular videos. At its core is a novel Motion-Adaptive Spline (MAS) method, which represents continuous dynamic 3D Gaussian trajectories using cubic Hermite splines with a small number of control points. For MAS, we introduce a Motion-Adaptive Control points Pruning (MACP) method to model the deformation of each dynamic 3D Gaussian across varying motions, progressively pruning control points while maintaining dynamic modeling integrity. Additionally, we present a joint optimization strategy for camera parameter estimation and 3D Gaussian attributes, leveraging photometric and geometric consistency. This eliminates the need for Structure-from-Motion preprocessing and enhances SplineGS's robustness in real-world conditions. Experiments show that SplineGS significantly outperforms state-of-the-art methods in novel view synthesis quality for dynamic scenes from monocular videos, achieving thousands times faster rendering speed.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。