通过匹配先验增强3D高斯点云结构一致性,提升稀疏视角下的新视角合成效果。
Structure Consistent Gaussian Splatting with Matching Prior for Few-shot Novel View Synthesis
- 引入射线绑定的高斯原型,约束其位置沿射线优化
- 在复杂大场景中实现优于现有方法的新视角合成质量
- 特别适合低数据量、大场景的3D重建任务
尽管新视角合成技术进展显著,基于神经辐射场(NeRF)或3D高斯点云(3DGS)的方法在输入稀疏时性能严重下降。现有方法虽有改进,但在大场景下仍难以高效生成高质量结果。本文提出SCGaussian,一种利用匹配先验的结构一致3D高斯点云方法,以学习一致的3D场景结构。针对高斯属性间的强耦合性,从渲染几何和高斯原型位置两方面优化场景结构,其中位置优化因原始3DGS缺乏结构特性而难以直接约束。为此,我们设计了一种混合高斯表示:除普通非结构高斯外,还引入与匹配射线绑定的高斯原型,其位置优化被限制在射线上。从而可利用匹配对应关系,直接引导这些高斯原型收敛至射线与表面的交点。在前向、环绕及复杂大场景上的大量实验表明,该方法达到当前最优性能并具有高效率。代码已开源于https://github.com/prstrive/SCGaussian。
原文摘要 · Abstract (English)
Despite the substantial progress of novel view synthesis, existing methods, either based on the Neural Radiance Fields (NeRF) or more recently 3D Gaussian Splatting (3DGS), suffer significant degradation when the input becomes sparse. Numerous efforts have been introduced to alleviate this problem, but they still struggle to synthesize satisfactory results efficiently, especially in the large scene. In this paper, we propose SCGaussian, a Structure Consistent Gaussian Splatting method using matching priors to learn 3D consistent scene structure. Considering the high interdependence of Gaussian attributes, we optimize the scene structure in two folds: rendering geometry and, more importantly, the position of Gaussian primitives, which is hard to be directly constrained in the vanilla 3DGS due to the non-structure property. To achieve this, we present a hybrid Gaussian representation. Besides the ordinary non-structure Gaussian primitives, our model also consists of ray-based Gaussian primitives that are bound to matching rays and whose optimization of their positions is restricted along the ray. Thus, we can utilize the matching correspondence to directly enforce the position of these Gaussian primitives to converge to the surface points where rays intersect. Extensive experiments on forward-facing, surrounding, and complex large scenes show the effectiveness of our approach with state-of-the-art performance and high efficiency. Code is available at https://github.com/prstrive/SCGaussian.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。