用轻量2.5D分层表示,让2D视频实现沉浸式观感。
CPSL: Representing Volumetric Video via Content-Promoted Scene Layers
- 按深度与内容显著性分解帧为分层,每层带软透明带和边缘深度缓存。
- 在多个基准上相比基线提升感知质量,存储与渲染成本降低数倍。
- 适合追求低延迟沉浸体验的实时视频通信场景。
体素视频支持自由视角探索和真实运动视差,带来沉浸式交互体验。然而,现有从显式点云到隐式神经场的体素表示在采集、计算和渲染上成本高昂,限制了其按需视频的可扩展性及实时通信可行性。为此,我们提出内容促进场景分层(CPSL),一种紧凑的2.5D视频表示,将体素视频的感知优势引入传统2D内容。基于每帧深度与内容显著性,CPSL将每帧分解为少量几何一致的分层,每层配备软透明带和边缘深度缓存,共同保持遮挡顺序与边界连续性。这些轻量级、可二维编码的资源通过深度加权扭曲与前后透明合成实现校正视差的新视角生成,避免昂贵的3D重建。时间上,采用运动引导传播与分层编码保持帧间一致性,支持标准视频编码器下的实时播放。在多个基准测试中,CPSL在感知质量与边界保真度上优于分层与神经场基线,同时存储与渲染成本降低数倍。本方法为从2D视频迈向可扩展的2.5D沉浸媒体提供了实用路径。
原文摘要 · Abstract (English)
Volumetric video enables immersive and interactive visual experiences by supporting free viewpoint exploration and realistic motion parallax. However, existing volumetric representations from explicit point clouds to implicit neural fields, remain costly in capture, computation, and rendering, which limits their scalability for on-demand video and reduces their feasibility for real-time communication. To bridge this gap, we propose Content-Promoted Scene Layers (CPSL), a compact 2.5D video representation that brings the perceptual benefits of volumetric video to conventional 2D content. Guided by per-frame depth and content saliency, CPSL decomposes each frame into a small set of geometry-consistent layers equipped with soft alpha bands and an edge-depth cache that jointly preserve occlusion ordering and boundary continuity. These lightweight, 2D-encodable assets enable parallax-corrected novel-view synthesis via depth-weighted warping and front-to-back alpha compositing, bypassing expensive 3D reconstruction. Temporally, CPSL maintains inter-frame coherence using motion-guided propagation and per-layer encoding, supporting real-time playback with standard video codecs. Across multiple benchmarks, CPSL achieves superior perceptual quality and boundary fidelity compared with layer-based and neural-field baselines while reducing storage and rendering cost by several folds. Our approach offer a practical path from 2D video to scalable 2.5D immersive media.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。