arXiv:2411.17044cs.CVcs.GR2024-11AAAI被引 14

用动态感知的锚点生长法,高效重建高保真动态场景。

4D Scaffold Gaussian Splatting with Dynamic-Aware Anchor Growing for Efficient and High-Fidelity Dynamic Scene Reconstruction

  • 以4D锚点压缩动态内容,通过MLP生成局部时空高斯
  • 动态区域锚点自动增加,重建质量显著提升
  • 适合需要高保真动态建模的视觉应用

通过4D高斯建模动态场景可实现高视觉保真度和快速渲染,但存储开销大。现有方法通过大幅减少高斯数量来降低代价,却不可避免地移除对高质量渲染至关重要的高斯,导致动态区域严重失真。本文提出一种新型4D锚点框架,不减少高斯数量,而是将足够多的高斯压缩为网格对齐的紧凑4D锚点特征。每个锚点由MLP生成一组神经4D高斯,代表局部时空区域。这些神经4D高斯以极少参数捕捉时间变化,适配基于MLP的生成机制。同时引入动态感知锚点生长策略,向重建不足的动态区域分配额外锚点。通过根据高斯的时间覆盖度调整累积梯度,显著提升动态区域重建质量。实验表明,本方法在动态区域达到领先视觉质量,优于所有基线,且存储成本实际可行。

原文摘要 · Abstract (English)

Modeling dynamic scenes through 4D Gaussians offers high visual fidelity and fast rendering speeds, but comes with significant storage overhead. Recent approaches mitigate this cost by aggressively reducing the number of Gaussians. However, this inevitably removes Gaussians essential for high-quality rendering, leading to severe degradation in dynamic regions. In this paper, we introduce a novel 4D anchor-based framework that tackles the storage cost in different perspective. Rather than reducing the number of Gaussians, our method retains a sufficient quantity to accurately model dynamic contents, while compressing them into compact, grid-aligned 4D anchor features. Each anchor is processed by an MLP to spawn a set of neural 4D Gaussians, which represent a local spatiotemporal region. We design these neural 4D Gaussians to capture temporal changes with minimal parameters, making them well-suited for the MLP-based spawning. Moreover, we introduce a dynamic-aware anchor growing strategy to effectively assign additional anchors to under-reconstructed dynamic regions. Our method adjusts the accumulated gradients with Gaussians' temporal coverage, significantly improving reconstruction quality in dynamic regions. Experimental results highlight that our method achieves state-of-the-art visual quality in dynamic regions, outperforming all baselines by a large margin with practical storage costs.

动态建模4D高斯场景重建

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。