从粗到细生成3D物体部件边界框,模拟人类创作思维过程。
BoxSplitGen: A Generative Model for 3D Part Bounding Boxes in Varying Granularity
- 通过迭代分裂粗略边界框,逐步细化3D部件结构。
- 在ShapeNet数据集上,生成精度优于传统令牌预测与补全方法。
- 适合需要渐进式设计的3D建模用户,如工业设计与游戏开发。
人类创造力遵循从抽象概念逐步细化为细节的感知过程。尽管3D生成模型已取得显著进展,但专门辅助人类在3D创作中从粗略到精细构建的模型仍缺乏探索。本文提出一种框架,通过迭代分裂边界框实现直观、交互式的3D形状生成。核心包含两个生成模型:盒分裂生成模型(BoxSplitGen)和盒到形状生成模型。前者利用凝聚合并生成的部件边界框,学习其逆过程——分裂序列,包含选择待分裂盒子的类别分布与生成两个新盒子的分布;后者基于现有3D扩散模型的形状先验,引入边界框条件进行适配。实验表明,盒分裂模型在无条件扩散模型的补全与令牌预测方法上表现更优;盒到形状模型基于先进3D扩散模型,生成质量超越先前工作。
原文摘要 · Abstract (English)
Human creativity follows a perceptual process, moving from abstract ideas to finer details during creation. While 3D generative models have advanced dramatically, models specifically designed to assist human imagination in 3D creation -- particularly for detailing abstractions from coarse to fine -- have not been explored. We propose a framework that enables intuitive and interactive 3D shape generation by iteratively splitting bounding boxes to refine the set of bounding boxes. The main technical components of our framework are two generative models: the box-splitting generative model and the box-to-shape generative model. The first model, named BoxSplitGen, generates a collection of 3D part bounding boxes with varying granularity by iteratively splitting coarse bounding boxes. It utilizes part bounding boxes created through agglomerative merging and learns the reverse of the merging process -- the splitting sequences. The model consists of two main components: the first learns the categorical distribution of the box to be split, and the second learns the distribution of the two new boxes, given the set of boxes and the indication of which box to split. The second model, the box-to-shape generative model, is trained by leveraging the 3D shape priors learned by an existing 3D diffusion model while adapting the model to incorporate bounding box conditioning. In our experiments, we demonstrate that the box-splitting generative model outperforms token prediction models and the inpainting approach with an unconditional diffusion model. Also, we show that our box-to-shape model, based on a state-of-the-art 3D diffusion model, provides superior results compared to a previous model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。