提出可变形动态场,高效分离真实场景中的移动物体与静态背景。
ForestSplats: Deformable transient field for Gaussian Splatting in the Wild
- 用可变形动态场建模每帧的动态元素,避免依赖大模型特征。
- 通过超像素掩码精准划分遮挡物边界,减少误判。
- 自适应稠密化机制防止在遮挡区生成冗余点云,节省内存。
最近,3D高斯点阵(3D-GS)在静态场景中展现出实时渲染速度和高质量效果。然而,在真实环境因存在动态物体、光照变化及不同程度遮挡时,其性能显著下降。现有方法依赖预训练视觉基础模型或额外动态场管道来估计遮挡物或动态元素,但存在两个缺陷:1)使用视觉基础模型的语义特征带来额外计算开销;2)动态场需大量内存存储每视图的高斯点,且仅靠光度误差难以明确界定遮挡边界。为此,我们提出ForestSplats,一种新颖方法,利用可变形动态场和超像素感知掩码,在无约束图像集合中高效表示二维场景的动态元素,并在不依赖视觉基础模型的前提下,有效将静态场景与动态干扰分离。我们设计了可变形动态场以捕捉每视图的动态元素,引入超像素感知掩码,结合光度误差与超像素信息,清晰定义遮挡边界。此外,提出不确定性感知稠密化策略,避免在遮挡边界内生成高斯点。在多个基准数据集上的实验表明,ForestSplats在无需视觉基础模型的情况下优于现有方法,且在表示动态元素时具有显著内存效率。
原文摘要 · Abstract (English)
Recently, 3D Gaussian Splatting (3D-GS) has emerged, showing real-time rendering speeds and high-quality results in static scenes. Although 3D-GS shows effectiveness in static scenes, their performance significantly degrades in real-world environments due to transient objects, lighting variations, and diverse levels of occlusion. To tackle this, existing methods estimate occluders or transient elements by leveraging pre-trained models or integrating additional transient field pipelines. However, these methods still suffer from two defects: 1) Using semantic features from the Vision Foundation model (VFM) causes additional computational costs. 2) The transient field requires significant memory to handle transient elements with per-view Gaussians and struggles to define clear boundaries for occluders, solely relying on photometric errors. To address these problems, we propose ForestSplats, a novel approach that leverages the deformable transient field and a superpixel-aware mask to efficiently represent transient elements in the 2D scene across unconstrained image collections and effectively decompose static scenes from transient distractors without VFM. We designed the transient field to be deformable, capturing per-view transient elements. Furthermore, we introduce a superpixel-aware mask that clearly defines the boundaries of occluders by considering photometric errors and superpixels. Additionally, we propose uncertainty-aware densification to avoid generating Gaussians within the boundaries of occluders during densification. Through extensive experiments across several benchmark datasets, we demonstrate that ForestSplats outperforms existing methods without VFM and shows significant memory efficiency in representing transient elements.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。