基于视觉显著性与元学习的3D高斯溅射视频自适应流媒体技术
Adaptive 3D Gaussian Splatting Video Streaming: Visual Saliency-Aware Tiling and Meta-Learning-Based Bitrate Adaptation
- 根据视觉显著性动态分块，每块含多质量层级和形变场
- 提出3D表示与2D渲染双域质量评估框架
- 元学习驱动码率自适应，跨网络条件表现更优
3D高斯溅射视频（3DGS）流媒体近年来在学术界和工业界成为研究热点，因其能提供沉浸式3D视频体验。然而该领域仍处于初期阶段，存在分块、质量评估和码率自适应等基础挑战亟待解决。本文提出一套完整解决方案：首先设计一种由显著性分析引导的自适应3DGS分块技术，融合空间与时间特征；每个分块编码为具有专用形变场和多质量层级的版本，支持自适应选择。其次，提出一种新型3DGS视频质量评估框架，联合评估流媒体过程中3D表示的空间域退化及生成2D渲染图像的质量。此外，开发了专为3DGS视频流媒体定制的元学习码率自适应算法，在不同网络条件下均实现最优性能。大量实验表明，所提方法显著优于现有先进方法。
原文摘要 · Abstract (English)
3D Gaussian splatting video (3DGS) streaming has recently emerged as a research hotspot in both academia and industry, owing to its impressive ability to deliver immersive 3D video experiences. However, research in this area is still in its early stages, and several fundamental challenges, such as tiling, quality assessment, and bitrate adaptation, require further investigation. In this paper, we tackle these challenges by proposing a comprehensive set of solutions. Specifically, we propose an adaptive 3DGS tiling technique guided by saliency analysis, which integrates both spatial and temporal features. Each tile is encoded into versions possessing dedicated deformation fields and multiple quality levels for adaptive selection. We also introduce a novel quality assessment framework for 3DGS video that jointly evaluates spatial-domain degradation in 3DGS representations during streaming and the quality of the resulting 2D rendered images. Additionally, we develop a meta-learning-based adaptive bitrate algorithm specifically tailored for 3DGS video streaming, achieving optimal performance across varying network conditions. Extensive experiments demonstrate that our proposed approaches significantly outperform state-of-the-art methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。