用2D高斯点表示视频，实现高效压缩与超快渲染。
GSVC: Efficient Video Representation and Compression Through 2D Gaussian Splatting
- 以相邻帧预测方式建模时间冗余，加速训练并提升压缩效率。
- 在1920x1080视频上实现1500帧/秒渲染速度，质量接近AV1/VVC。
- 自动删减低贡献点、动态添加新物体点，适合动态视频压缩场景。
3D高斯点已成为静态3D场景的高效学习表示。本文探索将2D高斯点作为视频的新表示原语，提出GSVC方法，学习一组2D高斯点以有效表示和压缩视频帧。该方法包含：(i) 利用相邻帧间的时间冗余，基于前一帧预测当前帧的高斯点，加快训练并提升压缩效率；(ii) 通过移除对画质贡献低的高斯点，控制文件大小与质量的权衡；(iii) 随机增加高斯点以捕捉大运动或新出现物体的动态；(iv) 基于学习过程中的损失差异检测关键帧。实验表明，GSVC在码率-失真权衡上达到与最新视频编码器（如AV1、VVC）相当的性能，并在1920x1080视频上实现1500帧/秒的渲染速度。
原文摘要 · Abstract (English)
3D Gaussian splats have emerged as a revolutionary, effective, learned representation for static 3D scenes. In this work, we explore using 2D Gaussian splats as a new primitive for representing videos. We propose GSVC, an approach to learning a set of 2D Gaussian splats that can effectively represent and compress video frames. GSVC incorporates the following techniques: (i) To exploit temporal redundancy among adjacent frames, which can speed up training and improve the compression efficiency, we predict the Gaussian splats of a frame based on its previous frame; (ii) To control the trade-offs between file size and quality, we remove Gaussian splats with low contribution to the video quality; (iii) To capture dynamics in videos, we randomly add Gaussian splats to fit content with large motion or newly-appeared objects; (iv) To handle significant changes in the scene, we detect key frames based on loss differences during the learning process. Experiment results show that GSVC achieves good rate-distortion trade-offs, comparable to state-of-the-art video codecs such as AV1 and VVC, and a rendering speed of 1500 fps for a 1920x1080 video.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。