用2D+3D高斯点分层建模动态与静态物体,提升视角合成质量。
HybridGS: Decoupling Transients and Statics with 2D and 3D Gaussian Splatting
- 2D高斯点捕捉单视图动态对象,3D高斯点建模整体静态场景
- 在基准数据集上实现室内室外场景的顶尖新视角合成效果
- 适合处理含干扰动态物体的复杂场景,如街景、室内活动
在包含动态物体的场景中生成高质量的新视角渲染结果,对3D高斯点积(3DGS)构成挑战。本文提出一种新型混合表示方法——HybridGS,采用2D高斯点表示每张图像中的动态物体,同时保留传统3D高斯点对整个静态场景进行建模。由于3DGS更适合假设多视角一致性的静态场景,而动态物体仅偶尔出现且不满足该假设,因此将其视为从单一视角出发的平面对象,使用2D高斯点建模。该方法从基本视角一致性角度分解场景,更具合理性。此外,我们设计了一种新的多视角协同监督机制,利用共可见区域信息,进一步强化动态与静态成分的区分。最后,提出一种简单但有效的多阶段训练策略,确保在多种设置下训练稳健且生成高质量视角。在多个基准数据集上的实验表明,本方法在室内外场景中均实现了当前最优的新视角合成性能,即使存在干扰元素也表现优异。
原文摘要 · Abstract (English)
Generating high-quality novel view renderings of 3D Gaussian Splatting (3DGS) in scenes featuring transient objects is challenging. We propose a novel hybrid representation, termed as HybridGS, using 2D Gaussians for transient objects per image and maintaining traditional 3D Gaussians for the whole static scenes. Note that, the 3DGS itself is better suited for modeling static scenes that assume multi-view consistency, but the transient objects appear occasionally and do not adhere to the assumption, thus we model them as planar objects from a single view, represented with 2D Gaussians. Our novel representation decomposes the scene from the perspective of fundamental viewpoint consistency, making it more reasonable. Additionally, we present a novel multi-view regulated supervision method for 3DGS that leverages information from co-visible regions, further enhancing the distinctions between the transients and statics. Then, we propose a straightforward yet effective multi-stage training strategy to ensure robust training and high-quality view synthesis across various settings. Experiments on benchmark datasets show our state-of-the-art performance of novel view synthesis in both indoor and outdoor scenes, even in the presence of distracting elements.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。