4DGC通过动态高斯表示与率失真优化,实现高效可流式传输的自由视角视频。
4DGC: Rate-Aware 4D Gaussian Compression for Efficient Streamable Free-Viewpoint Video
- 引入运动感知的动态高斯表示,用紧凑运动网格和稀疏补偿高斯减少冗余。
- 端到端压缩框架支持可变码率,显著降低存储量并保持高质量重建。
- 适合需要低延迟、高画质的自由视角视频应用,如VR直播与云渲染。
3D高斯点阵(3DGS)在实现照片级真实感自由视角视频(FVV)方面潜力巨大,但大量高斯点及其属性带来存储与传输挑战。现有方法通常将动态3DGS表示与压缩分开处理,忽略运动信息及率失真(RD)权衡,导致性能下降与模型冗余。为此,我们提出4DGC——一种新型率感知4D高斯压缩框架,大幅降低存储空间并保持优异的RD性能。具体而言,4DGC引入运动感知的动态高斯表示,利用紧凑运动网格与稀疏补偿高斯点,捕捉帧间相似性,有效处理大运动,提升质量并减少时序冗余。此外,我们设计端到端压缩方案,采用可微量化与微型隐式熵模型,高效压缩运动网格与补偿高斯点。整个框架基于率失真权衡联合优化。大量实验表明,4DGC支持可变码率,在多个数据集上持续优于现有方法。
原文摘要 · Abstract (English)
3D Gaussian Splatting (3DGS) has substantial potential for enabling photorealistic Free-Viewpoint Video (FVV) experiences. However, the vast number of Gaussians and their associated attributes poses significant challenges for storage and transmission. Existing methods typically handle dynamic 3DGS representation and compression separately, neglecting motion information and the rate-distortion (RD) trade-off during training, leading to performance degradation and increased model redundancy. To address this gap, we propose 4DGC, a novel rate-aware 4D Gaussian compression framework that significantly reduces storage size while maintaining superior RD performance for FVV. Specifically, 4DGC introduces a motion-aware dynamic Gaussian representation that utilizes a compact motion grid combined with sparse compensated Gaussians to exploit inter-frame similarities. This representation effectively handles large motions, preserving quality and reducing temporal redundancy. Furthermore, we present an end-to-end compression scheme that employs differentiable quantization and a tiny implicit entropy model to compress the motion grid and compensated Gaussians efficiently. The entire framework is jointly optimized using a rate-distortion trade-off. Extensive experiments demonstrate that 4DGC supports variable bitrates and consistently outperforms existing methods in RD performance across multiple datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。