为3D高斯点云设计自适应分层编码,显著提升低码率下的渲染质量。
Region-Adaptive Learned Hierarchical Encoding for 3D Gaussian Splatting Data
- 基于八叉树结构构建多分辨率分层隐变量,适配3DGS不规则分布特性。
- 在1MB以下码率下,渲染峰值信噪比最高提升2dB,优于现有方法。
- 适合对带宽敏感的3D媒体流场景,如移动设备实时渲染。
我们提出区域自适应学习分层编码(RALHE)用于3D高斯点云(3DGS)数据压缩。尽管3DGS在新视角合成中广受欢迎,但模型尺寸限制了其在带宽受限场景(如体素媒体流)中的部署。为此,我们设计了一种学习型分层隐表示,借鉴“过拟合”式图像压缩(如Cool-Chic和C3)的思想,高效编码3DGS属性。与图像不同,3DGS数据具有不规则的空间分布(几何)且包含多个定义在非规则几何上的属性信号。我们的编码器利用体素化3DGS几何的八叉树结构,获得多层次分辨率表示。在全局码率约束下,对每个高斯属性过拟合隐变量,通过轻量解码网络独立解码。训练时采用自回归概率模型,利用八叉树推导出的上下文估算比特率。多分辨率隐变量、解码器及自回归熵编码网络针对每个高斯属性联合优化。实验表明,所提的RALHE压缩框架在低于1MB的低码率下,相较基线3DGS压缩方法,渲染峰值信噪比最高提升2dB。
原文摘要 · Abstract (English)
We introduce Region-Adaptive Learned Hierarchical Encoding (RALHE) for 3D Gaussian Splatting (3DGS) data. While 3DGS has recently become popular for novel view synthesis, the size of trained models limits its deployment in bandwidth-constrained applications such as volumetric media streaming. To address this, we propose a learned hierarchical latent representation that builds upon the principles of "overfitted" learned image compression (e.g., Cool-Chic and C3) to efficiently encode 3DGS attributes. Unlike images, 3DGS data have irregular spatial distributions of Gaussians (geometry) and consist of multiple attributes (signals) defined on the irregular geometry. Our codec is designed to account for these differences between images and 3DGS. Specifically, we leverage the octree structure of the voxelized 3DGS geometry to obtain a hierarchical multi-resolution representation. Our approach overfits latents to each Gaussian attribute under a global rate constraint. These latents are decoded independently through a lightweight decoder network. To estimate the bitrate during training, we employ an autoregressive probability model that leverages octree-derived contexts from the 3D point structure. The multi-resolution latents, decoder, and autoregressive entropy coding networks are jointly optimized for each Gaussian attribute. Experiments demonstrate that the proposed RALHE compression framework achieves a rendering PSNR gain of up to 2dB at low bitrates (less than 1 MB) compared to the baseline 3DGS compression methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。