融合可变形高斯与动态神经表面，实现单目动态场景三维重建
DGNS: Deformable Gaussian Splatting and Dynamic Neural Surface for Monocular Dynamic 3D Reconstruction
- 用可变形高斯点云引导光线采样并提供深度监督
- 在公开数据集上达到最优3D重建效果，新视角合成性能优异
- 适合做动态场景重建、视觉里程计或机器人感知的研究者
从单目视频中进行动态场景重建对真实应用至关重要。本文提出DGNS，一种结合可变形高斯点云（Deformable Gaussian Splatting）与动态神经表面（Dynamic Neural Surface）的混合框架，可同时实现动态新视角生成与三维几何重建。训练过程中，可变形高斯模块生成的深度图用于指导光线采样，加快处理速度，并为动态神经表面模块提供深度监督，提升几何重建精度；反之，动态神经表面则指导高斯原型在表面周围的分布，改善渲染质量。此外，提出一种深度滤波方法进一步优化深度监督。在多个公开数据集上的大量实验表明，DGNS在3D重建方面达到当前最优性能，且在新视角合成任务中表现竞争力。
原文摘要 · Abstract (English)
Dynamic scene reconstruction from monocular video is essential for real-world applications. We introduce DGNS, a hybrid framework integrating \underline{D}eformable \underline{G}aussian Splatting and Dynamic \underline{N}eural \underline{S}urfaces, effectively addressing dynamic novel-view synthesis and 3D geometry reconstruction simultaneously. During training, depth maps generated by the deformable Gaussian splatting module guide the ray sampling for faster processing and provide depth supervision within the dynamic neural surface module to improve geometry reconstruction. Conversely, the dynamic neural surface directs the distribution of Gaussian primitives around the surface, enhancing rendering quality. In addition, we propose a depth-filtering approach to further refine depth supervision. Extensive experiments conducted on public datasets demonstrate that DGNS achieves state-of-the-art performance in 3D reconstruction, along with competitive results in novel-view synthesis.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。