用3D高斯点云生成复杂盆景,结构更真实
3DBonsai: Structure-Aware Bonsai Modeling Using Conditioned 3D Gaussian Splatting
- 设计可训练的3D空间殖民算法生成盆景骨架
- 在真实数据集上生成效果优于现有方法
- 适合对中式盆景生成感兴趣的创作者
近期文本到3D生成方法借助2D扩散模型与3D先验取得了显著进展,但现有方法依赖的3D先验缺乏细节和复杂结构信息,难以生成如盆景这类复杂形态。本文提出3DBonsai,一种新的文本到3D盆景生成框架。技术上,我们首先设计一个可训练的3D空间殖民算法生成盆景结构,并通过随机采样与点云增强构建3D高斯先验。提出两种生成路径:细粒度结构条件生成,利用3D结构先验初始化3D高斯以生成精细复杂盆景;粗粒度结构条件生成,采用多视角结构一致性模块对齐2D与3D结构。此外,我们构建了统一的中式盆景2D/3D数据集。实验表明,3DBonsai显著优于现有方法,为结构感知的3D盆景生成树立新基准。
原文摘要 · Abstract (English)
Recent advancements in text-to-3D generation have shown remarkable results by leveraging 3D priors in combination with 2D diffusion. However, previous methods utilize 3D priors that lack detailed and complex structural information, limiting them to generating simple objects and presenting challenges for creating intricate structures such as bonsai. In this paper, we propose 3DBonsai, a novel text-to-3D framework for generating 3D bonsai with complex structures. Technically, we first design a trainable 3D space colonization algorithm to produce bonsai structures, which are then enhanced through random sampling and point cloud augmentation to serve as the 3D Gaussian priors. We introduce two bonsai generation pipelines with distinct structural levels: fine structure conditioned generation, which initializes 3D Gaussians using a 3D structure prior to produce detailed and complex bonsai, and coarse structure conditioned generation, which employs a multi-view structure consistency module to align 2D and 3D structures. Moreover, we have compiled a unified 2D and 3D Chinese-style bonsai dataset. Our experimental results demonstrate that 3DBonsai significantly outperforms existing methods, providing a new benchmark for structure-aware 3D bonsai generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。