用文本引导从点云生长3D高斯,提升生成几何精度
GaussianGrow: Geometry-aware Gaussian Growing from 3D Point Clouds with Text Guidance

- 从点云出发,通过文本引导逐步生长3D高斯,保证几何一致性
- 多视角扩散模型合成一致外观,支持真实扫描点云生成
- 自动定位未覆盖区域并迭代补全,实现完整高斯体生成
3D高斯泼溅在渲染效率与质量上表现优异,但缺乏几何先验时生成3D高斯仍具挑战。现有方法尝试预测点图作为几何参考以推断高斯基元,但估计的几何不可靠会引发生成质量下降。本文提出GaussianGrow,一种从易获取的3D点云出发,学习逐步生长3D高斯的新方法,天然保证生成过程的几何准确性。我们设计了文本引导的高斯生长机制,利用多视角扩散模型从输入点云中合成一致外观以提供监督信号。为缓解不同视角融合带来的伪影,我们在重叠区域中识别非预设相机位姿生成的新视图施加约束。针对难以观测区域,提出通过检测点云中最大未生长区域并使用预训练2D扩散模型对渲染视图进行修复,实现迭代补全。该过程持续进行直至完整高斯体生成。我们在合成及真实扫描点云上进行了广泛评估,验证了方法的有效性。
原文摘要 · Abstract (English)
3D Gaussian Splatting has demonstrated superior performance in rendering efficiency and quality, yet the generation of 3D Gaussians still remains a challenge without proper geometric priors. Existing methods have explored predicting point maps as geometric references for inferring Gaussian primitives, while the unreliable estimated geometries may lead to poor generations. In this work, we introduce GaussianGrow, a novel approach that generates 3D Gaussians by learning to grow them from easily accessible 3D point clouds, naturally enforcing geometric accuracy in Gaussian generation. Specifically, we design a text-guided Gaussian growing scheme that leverages a multi-view diffusion model to synthesize consistent appearances from input point clouds for supervision. To mitigate artifacts caused by fusing neighboring views, we constrain novel views generated at non-preset camera poses identified in overlapping regions across different views. For completing the hard-to-observe regions, we propose to iteratively detect the camera pose by observing the largest un-grown regions in point clouds and inpainting them by inpainting the rendered view with a pretrained 2D diffusion model. The process continues until complete Gaussians are generated. We extensively evaluate GaussianGrow on text-guided Gaussian generation from synthetic and even real-scanned point clouds. Project Page: https://weiqi-zhang.github.io/GaussianGrow
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。