为3D高斯点阵实时传输设计新系统，提升画质并降低带宽消耗。
GSStream: 3D Gaussian Splatting based Volumetric Scene Streaming System
- 引入协同视口预测，结合多用户历史行为预判观看方向。
- 采用深度强化学习动态调整码率，适应复杂网络环境变化。
- 首个面向体场景的视口轨迹数据集，支持训练与仿真测试。
近期，3D高斯点阵（3DGS）技术在实时辐射场渲染中取得突破，带来沉浸式体验，但其带来的海量数据对带宽要求极高。尽管已有研究尝试通过多种方法压缩3DGS表示，实现实时分发仍具挑战。本文提出GSStream，一种支持3DGS数据格式的体场景流媒体系统。该系统集成协同视口预测模块，通过学习多用户历史序列中的共同先验与个体先验，更精准预测用户未来视角；同时引入基于深度强化学习的码率自适应模块，有效应对状态与动作空间高度变异的难题。此外，我们首次构建了面向体场景的用户视口轨迹数据集，用于模型训练与流媒体仿真。大量实验表明，相比现有主流体场景流系统，GSStream在视觉质量与网络利用率方面均表现更优。
原文摘要 · Abstract (English)
Recently, the 3D Gaussian splatting (3DGS) technique for real-time radiance field rendering has revolutionized the field of volumetric scene representation, providing users with an immersive experience. But in return, it also poses a large amount of data volume, which is extremely bandwidth-intensive. Cutting-edge researchers have tried to introduce different approaches and construct multiple variants for 3DGS to obtain a more compact scene representation, but it is still challenging for real-time distribution. In this paper, we propose GSStream, a novel volumetric scene streaming system to support 3DGS data format. Specifically, GSStream integrates a collaborative viewport prediction module to better predict users' future behaviors by learning collaborative priors and historical priors from multiple users and users' viewport sequences and a deep reinforcement learning (DRL)-based bitrate adaptation module to tackle the state and action space variability challenge of the bitrate adaptation problem, achieving efficient volumetric scene delivery. Besides, we first build a user viewport trajectory dataset for volumetric scenes to support the training and streaming simulation. Extensive experiments prove that our proposed GSStream system outperforms existing representative volumetric scene streaming systems in visual quality and network usage. Demo video: https://youtu.be/3WEe8PN8yvA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。