提出首个可流式传输的长时自由视角视频重建框架,抗误差且无需分组编码。
SoLAR: Error-Resilient Streamable Long-Horizon Free-Viewpoint Video Reconstruction with Anchor Activation and Latent Recalibration

- 动态激活关键锚点,自适应建模非刚性运动变化。
- 通过潜在差异重校准机制,显著抑制误差传播,保持低存储开销。
- 适合沉浸式媒体系统部署,尤其适用于长序列视频场景。
自由视角视频(FVV)已成为下一代沉浸式媒体系统的核心技术,受到广泛关注。以往方法主要针对短时视频序列,处理长时自由视角视频(LFVV)时性能显著下降。受比特分配理论启发,本文在率失真优化框架下分析基于动态锚点的体素化视频表示,提出首个具备误差鲁棒性的可流式传输FVV框架SoLAR,可在不进行图像组(GOP)分组的情况下稳定保持长序列重建质量。提出锚点激活动态(AAD),使动态锚点能通过激活信息量高的锚点并抑制冗余锚点来建模非刚性形变。进一步引入潜在差异感知重校准(LaDAR),识别潜在表示间的差异并重校准网络中编码的对应关系,有效缓解了LFVV中的误差传播,同时不牺牲实时性或存储紧凑性。大量实验表明,SoLAR在保持最小存储开销的同时达到最先进的重建性能,为FVV重建提供了新方向,并推动了沉浸式系统的实际应用。演示视频见补充材料。
原文摘要 · Abstract (English)
Free-Viewpoint Video (FVV) has emerged as a cornerstone of next-generation immersive media systems and attracted widespread attention. Previous methods primarily focus on short video sequences and suffer from significant performance degradation when processing long-horizon free-viewpoint video (LFVV). Motivated by bit allocation theory, we analyze dynamic-anchor-based volumetric video representation within a rate-distortion optimization framework and propose \textbf{SoLAR}, which is the first error-resilient streamable FVV framework that maintains stable reconstruction quality on long sequences without requiring group-of-pictures partitioning. We propose the Anchor Activation Dynamics (AAD), which enables dynamic anchors to model non-rigid transformations by dynamically activating informative anchors and suppressing redundant ones. Furthermore, we introduce Latent Discrepancy Aware Recalibration (LaDAR), which is a mechanism to identify discrepancies between latent representations and recalibrate the correspondences encoded in the network, effectively mitigating error propagation in LFVV without compromising real-time performance or storage compactness. Extensive experiments demonstrate that \textbf{SoLAR} achieves state-of-the-art reconstruction performance while maintaining minimum storage overhead, which provides a new direction for LFVV reconstruction and advances the practical deployment of immersive systems. Demo free-viewpoint videos are provided in the supplementary material.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。