arXiv:2409.16615cs.CV2024-09被引 2

用形变建模提升3D视频流带宽效率与画质,适应网络波动。

DeformStream: Deformation-based Adaptive Volumetric Video Streaming

  • 基于网格形变重建帧,利用时序冗余减少传输量。
  • 在动态网络下实现带宽与画质的最优平衡,性能超越现有方法。
  • 适合实时3D视频应用,尤其对网络不稳定场景友好。

体视频流提供沉浸式3D体验,但高带宽需求和延迟问题限制了其实时传输。传统点云流在缩放时损失画质,神经渲染计算开销过大;虽网格流能保留表面细节与连接性,但传统方法逐帧传输,未能充分利用帧间时序冗余,导致带宽效率低且难以适应网络波动。本文提出基于形变的自适应体视频流框架DeformStream,利用网格表示的固有可变形性,通过嵌入形变重建后续帧,显著降低传输开销并保持帧间视觉连贯性。为缓解重建延迟与网络适应性问题,设计新QoE模型,考虑客户端形变延迟,并提出动态规划算法优化不同网络条件下视觉质量与带宽消耗的权衡。评估表明,DeformStream在带宽效率与画质上均优于现有网格流系统,为实时体视频应用提供了鲁棒解决方案。

原文摘要 · Abstract (English)

Volumetric video streaming offers immersive 3D experiences but faces significant challenges due to high bandwidth requirements and latency issues in transmitting detailed content in real time. Traditional methods like point cloud streaming compromise visual quality when zoomed in, and neural rendering techniques are too computationally intensive for real-time use. Though mesh-based streaming stands out by preserving surface detail and connectivity, offering a more refined representation for 3D content, traditional mesh streaming methods typically transmit data on a per-frame basis, failing to take full advantage of temporal redundancies across frames. This results in inefficient bandwidth usage and poor adaptability to fluctuating network conditions. We introduce Deformation-based Adaptive Volumetric Video Streaming, a novel framework that enhances volumetric video streaming performance by leveraging the inherent deformability of mesh-based representations. DeformStream uses embedded deformation to reconstruct subsequent frames from inter-frame motion, significantly reducing bandwidth usage while ensuring visual coherence between frames. To address frame reconstruction overhead and network adaptability, we formulate a new QoE model that accounts for client-side deformation latency and design a dynamic programming algorithm to optimize the trade-off between visual quality and bandwidth consumption under varying network conditions. Our evaluation demonstrates that Deformation-based Adaptive Volumetric Video Streaming outperforms existing mesh-based streaming systems in both bandwidth efficiency and visual quality, offering a robust solution for real-time volumetric video applications.

3D视频网格流形变建模自适应传输

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。