提出显式4D高斯点云,实现动态场景快速高质量渲染。
Fully Explicit Dynamic Gaussian Splatting
- 分离静态与动态高斯点,稀疏采样时空位置并插值生成连续运动。
- 在单张2080Ti上实现62帧/秒的实时渲染,优于现有方法。
- 适合需要高效动态3D重建的科研与工业应用。
3D高斯点云通过密集3D先验和显式表示,在静态场景中实现了快速且高质量的渲染。然而,这些先验和表示在动态运动的新视角合成中反而成为瓶颈,主要因需增加训练与渲染时间以建模动态变化。本文提出显式4D高斯点云(Ex4DGS):训练时分离静态与动态高斯点,并在稀疏时间戳上显式采样动态点的位置与旋转;通过插值实现时空连续的物体运动表达,同时降低计算开销。此外,引入渐进式训练策略与点回溯技术,提升收敛性。先用短时间跨度训练,逐步扩展时间范围,适配少量点云数据;点回溯用于量化各高斯点随时间累积误差,从而检测并剔除动态场景中的错误点。在多种场景上的全面实验表明,该方法达到当前最优渲染质量,在单张2080Ti GPU上实现62帧/秒的快速渲染。
原文摘要 · Abstract (English)
3D Gaussian Splatting has shown fast and high-quality rendering results in static scenes by leveraging dense 3D prior and explicit representations. Unfortunately, the benefits of the prior and representation do not involve novel view synthesis for dynamic motions. Ironically, this is because the main barrier is the reliance on them, which requires increasing training and rendering times to account for dynamic motions. In this paper, we design a Explicit 4D Gaussian Splatting(Ex4DGS). Our key idea is to firstly separate static and dynamic Gaussians during training, and to explicitly sample positions and rotations of the dynamic Gaussians at sparse timestamps. The sampled positions and rotations are then interpolated to represent both spatially and temporally continuous motions of objects in dynamic scenes as well as reducing computational cost. Additionally, we introduce a progressive training scheme and a point-backtracking technique that improves Ex4DGS's convergence. We initially train Ex4DGS using short timestamps and progressively extend timestamps, which makes it work well with a few point clouds. The point-backtracking is used to quantify the cumulative error of each Gaussian over time, enabling the detection and removal of erroneous Gaussians in dynamic scenes. Comprehensive experiments on various scenes demonstrate the state-of-the-art rendering quality from our method, achieving fast rendering of 62 fps on a single 2080Ti GPU.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。