提出分层点式潜在表示,高效压缩动态高斯喷溅视频流。
HPC: Hierarchical Point-based Latent Representation for Streaming Dynamic Gaussian Splatting Compression
- 采用分层点式潜在表示,按每个高斯点建模避免空域冗余。
- 相比基线减少67%存储空间,重建质量保持高水平。
- 首次挖掘帧间参数相关性,实现神经网络端到端压缩。
动态高斯喷溅虽推动了自由视角视频的发展,但如何在小内存占用下维持高质量渲染并实现高效流媒体传输仍是挑战。现有方法多依赖潜在表示驱动神经网络预测帧间高斯残差,其潜在表示分为结构化网格与非结构化点两类。前者因建模未占用空间导致参数冗余,后者则因忽略局部相关性而紧凑性不足。为此,本文提出HPC框架,采用基于高斯点的分层点式潜在表示,避免空域参数冗余;通过定制聚合策略,实现低空间冗余下的高紧凑性。为进一步提升压缩效率,首次探索利用帧间参数相关性压缩用于流媒体动态高斯喷溅的神经网络。结合潜在表示压缩,构建全端到端压缩框架。大量实验表明,HPC显著优于当前最优方法,在保持高重建保真度的前提下,相较基线实现67%的存储降低。
原文摘要 · Abstract (English)
While dynamic Gaussian Splatting has driven significant advances in free-viewpoint video, maintaining its rendering quality with a small memory footprint for efficient streaming transmission still presents an ongoing challenge. Existing streaming dynamic Gaussian Splatting compression methods typically leverage a latent representation to drive the neural network for predicting Gaussian residuals between frames. Their core latent representations can be categorized into structured grid-based and unstructured point-based paradigms. However, the former incurs significant parameter redundancy by inevitably modeling unoccupied space, while the latter suffers from limited compactness as it fails to exploit local correlations. To relieve these limitations, we propose HPC, a novel streaming dynamic Gaussian Splatting compression framework. It employs a hierarchical point-based latent representation that operates on a per-Gaussian basis to avoid parameter redundancy in unoccupied space. Guided by a tailored aggregation scheme, these latent points achieve high compactness with low spatial redundancy. To improve compression efficiency, we further undertake the first investigation to compress neural networks for streaming dynamic Gaussian Splatting through mining and exploiting the inter-frame correlation of parameters. Combined with latent compression, this forms a fully end-to-end compression framework. Comprehensive experimental evaluations demonstrate that HPC substantially outperforms state-of-the-art methods. It achieves a storage reduction of 67% against its baseline while maintaining high reconstruction fidelity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。