4DGCPro实现高质量360度视频逐级流传输,手机端实时解码。
4DGCPro: Efficient Hierarchical 4D Gaussian Compression for Progressive Volumetric Video Streaming
- 分层4D高斯表示结合运动感知分组,减少时间冗余
- 单比特流支持多速率灵活切换,移动端实现实时渲染
- 端到端熵优化训练,压缩效率优于现有方法
实现与2D视频体验媲美的高保真体感视频无缝观看仍是开放挑战。现有体感视频压缩方法要么无法在单一模型内灵活调节质量与码率以适应不同网络和设备,要么难以在轻量级移动平台实现实时解码与渲染。为此,我们提出4DGCPro,一种新型分层4D高斯压缩框架,通过单一比特流实现体感视频的渐进式流传输,支持移动端实时解码与高质量渲染。具体地,我们设计了一种感知加权且适合压缩的分层4D高斯表示,结合运动感知自适应分组,有效降低时间冗余、保持一致性,并支持多层级细节渐进传输。此外,我们提出端到端熵优化训练方案,包含逐层率失真(RD)监督与属性特定熵建模,实现高效比特流生成。大量实验表明,4DGCPro可在单一模型中灵活调节质量与多码率,移动端实现实时解码与渲染,且在多个数据集上均优于现有方法的率失真性能。
原文摘要 · Abstract (English)
Achieving seamless viewing of high-fidelity volumetric video, comparable to 2D video experiences, remains an open challenge. Existing volumetric video compression methods either lack the flexibility to adjust quality and bitrate within a single model for efficient streaming across diverse networks and devices, or struggle with real-time decoding and rendering on lightweight mobile platforms. To address these challenges, we introduce 4DGCPro, a novel hierarchical 4D Gaussian compression framework that facilitates real-time mobile decoding and high-quality rendering via progressive volumetric video streaming in a single bitstream. Specifically, we propose a perceptually-weighted and compression-friendly hierarchical 4D Gaussian representation with motion-aware adaptive grouping to reduce temporal redundancy, preserve coherence, and enable scalable multi-level detail streaming. Furthermore, we present an end-to-end entropy-optimized training scheme, which incorporates layer-wise rate-distortion (RD) supervision and attribute-specific entropy modeling for efficient bitstream generation. Extensive experiments show that 4DGCPro enables flexible quality and multiple bitrate within a single model, achieving real-time decoding and rendering on mobile devices while outperforming existing methods in RD performance across multiple datasets. Project Page: https://mediax-sjtu.github.io/4DGCPro
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。