arXiv:2410.13613cs.CVcs.GR2024-10ICCV被引 48

用更少的高斯点实现动态3D场景高效渲染,内存压缩近200倍。

MEGA: Memory-Efficient 4D Gaussian Splatting for Dynamic Scenes

  • 将颜色分解为3参数直接色与轻量预测器,取代传统144参数球谐系数。
  • 通过熵约束变形场控制高斯点数量,仅用少量点即可拟合动态场景。
  • 在多个数据集上实现190倍以上存储压缩,仍保持高质量实时渲染。

4D高斯溅射(4DGS)近期成为高保真捕捉复杂动态3D场景的有力方法,利用4维高斯表示和友好的GPU光栅化器实现快速渲染。然而,4DGS面临显著挑战:需数百万个4维高斯点,每个包含大量属性,导致巨大内存与存储开销。本文提出一种内存高效的4DGS框架。我们通过将颜色属性分解为仅含3个参数的逐高斯直接颜色分量与共享的轻量交流色预测器,消除传统4DGS中高达144个参数的球谐系数需求,从而构建更紧凑的4维高斯表示。此外,引入熵约束的高斯形变技术,使用形变场扩展每个高斯点的作用范围,并集成基于透明度的熵损失以限制高斯点数量,促使模型尽可能少地使用高斯点来准确拟合动态场景。结合半精度存储与zip压缩,本框架在Technicolor与Neural 3D Video数据集上的存储量分别较原始4DGS减少约190倍和125倍,同时保持相当的渲染速度与场景重建质量,树立了该领域的全新标准。代码已开源:https://github.com/Xinjie-Q/MEGA。

原文摘要 · Abstract (English)

4D Gaussian Splatting (4DGS) has recently emerged as a promising technique for capturing complex dynamic 3D scenes with high fidelity. It utilizes a 4D Gaussian representation and a GPU-friendly rasterizer, enabling rapid rendering speeds. Despite its advantages, 4DGS faces significant challenges, notably the requirement of millions of 4D Gaussians, each with extensive associated attributes, leading to substantial memory and storage cost. This paper introduces a memory-efficient framework for 4DGS. We streamline the color attribute by decomposing it into a per-Gaussian direct color component with only 3 parameters and a shared lightweight alternating current color predictor. This approach eliminates the need for spherical harmonics coefficients, which typically involve up to 144 parameters in classic 4DGS, thereby creating a memory-efficient 4D Gaussian representation. Furthermore, we introduce an entropy-constrained Gaussian deformation technique that uses a deformation field to expand the action range of each Gaussian and integrates an opacity-based entropy loss to limit the number of Gaussians, thus forcing our model to use as few Gaussians as possible to fit a dynamic scene well. With simple half-precision storage and zip compression, our framework achieves a storage reduction by approximately 190$\times$ and 125$\times$ on the Technicolor and Neural 3D Video datasets, respectively, compared to the original 4DGS. Meanwhile, it maintains comparable rendering speeds and scene representation quality, setting a new standard in the field. Code is available at https://github.com/Xinjie-Q/MEGA.

4D高斯动态场景内存优化高效渲染

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。