用特征流提升4D高斯沉浸视频的压缩与渲染效率
GIFStream: 4D Gaussian-based Immersive Video with Feature Stream
- 用规范空间加时变特征流建模动态场景
- 30 Mbps下实现高质量实时渲染与快速解码
- 适合做沉浸式视频系统开发的研究者
沉浸式视频提供自由6-DoF观看体验，有望成为未来视频技术的关键。近期，4D高斯点阵因其高效渲染和高质量表现受到关注，但保持质量的同时控制存储仍具挑战。为此，我们提出GIFStream，一种基于规范空间和形变场的新型4D高斯表示，引入时变特征流以实现复杂运动建模，并通过时间对应性与运动感知剪枝实现高效压缩。同时，集成时空压缩网络实现端到端压缩。实验表明，GIFStream在30 Mbps下可实现高质量沉浸式视频，支持RTX 4090上的实时渲染与快速解码。
原文摘要 · Abstract (English)
Immersive video offers a 6-Dof-free viewing experience, potentially playing a key role in future video technology. Recently, 4D Gaussian Splatting has gained attention as an effective approach for immersive video due to its high rendering efficiency and quality, though maintaining quality with manageable storage remains challenging. To address this, we introduce GIFStream, a novel 4D Gaussian representation using a canonical space and a deformation field enhanced with time-dependent feature streams. These feature streams enable complex motion modeling and allow efficient compression by leveraging temporal correspondence and motion-aware pruning. Additionally, we incorporate both temporal and spatial compression networks for end-to-end compression. Experimental results show that GIFStream delivers high-quality immersive video at 30 Mbps, with real-time rendering and fast decoding on an RTX 4090. Project page: https://xdimlab.github.io/GIFStream
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。