用可逆变形网络实现长视频动态场景的高效视图合成。
HoliGS: Holistic Gaussian Splatting for Embodied View Synthesis
- 将场景分背景与动态物体,用高斯点建模并分层变形
- 在长视频上训练速度更快,渲染效率提升显著
- 适合需要多视角交互的真实世界动态场景重建
我们提出HoliGS,一种新型可变形高斯点阵框架,用于从长时间单目RGB视频中实现具身视图合成。相比先前的4D高斯点阵和动态NeRF方法在分钟级视频上存在训练开销大的问题,本方法利用可逆高斯点阵变形网络,准确重建大规模动态环境。具体地,将每个场景分解为静态背景与随时间变化的物体,分别由学习到的高斯原语表示,其经历全局刚性变换、骨骼驱动的关节运动以及通过可逆神经流实现的微小非刚性形变。这种分层形变策略可通过附着于完整规范前景形状(如第一人称或第三人称跟随)的高斯点,在多种具身相机轨迹下实现鲁棒的自由视角新视图渲染,即使存在大幅视角变化和多主体交互也表现良好。实验表明,该方法在挑战性数据集上达到更优重建质量,同时显著降低训练与渲染时间,相较于当前最先进的单目可变形NeRF具有明显优势。结果凸显了其在真实场景中具身视图合成的实用性与可扩展性。代码将公开。
原文摘要 · Abstract (English)
We propose HoliGS, a novel deformable Gaussian splatting framework that addresses embodied view synthesis from long monocular RGB videos. Unlike prior 4D Gaussian splatting and dynamic NeRF pipelines, which struggle with training overhead in minute-long captures, our method leverages invertible Gaussian Splatting deformation networks to reconstruct large-scale, dynamic environments accurately. Specifically, we decompose each scene into a static background plus time-varying objects, each represented by learned Gaussian primitives undergoing global rigid transformations, skeleton-driven articulation, and subtle non-rigid deformations via an invertible neural flow. This hierarchical warping strategy enables robust free-viewpoint novel-view rendering from various embodied camera trajectories by attaching Gaussians to a complete canonical foreground shape (\eg, egocentric or third-person follow), which may involve substantial viewpoint changes and interactions between multiple actors. Our experiments demonstrate that \ourmethod~ achieves superior reconstruction quality on challenging datasets while significantly reducing both training and rendering time compared to state-of-the-art monocular deformable NeRFs. These results highlight a practical and scalable solution for EVS in real-world scenarios. The source code will be released.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。