用离散扩散模型生成稀疏多类别3D体素结构，突破内存与类别不平衡瓶颈
Scaffold Diffusion: Sparse Multi-Category Voxel Structure Generation with Discrete Diffusion
- 将体素视为令牌，用离散扩散语言模型生成三维空间结构
- 在超过98%稀疏的数据上训练仍能生成真实连贯的方块房屋结构
- 支持交互式可视化生成过程，适合3D生成与游戏内容设计研究者
由于体素结构存在立方体内存增长问题，且稀疏性导致显著类别不平衡，生成真实的稀疏多类别3D体素结构极具挑战。本文提出Scaffold Diffusion，一种专为稀疏多类别3D体素结构设计的生成模型。通过将体素视为令牌，Scaffold Diffusion采用离散扩散语言模型生成3D体素结构。实验表明，离散扩散语言模型可拓展至非序列化的空间生成任务，实现空间一致性。在3D-Craft数据集的Minecraft房屋结构上评估，相较于以往基线和自回归方法，Scaffold Diffusion即使在超过98%稀疏的数据上训练，仍能生成真实且连贯的结构。提供交互式可视化工具：https://scaffold.deepexploration.org/
原文摘要 · Abstract (English)
Generating realistic sparse multi-category 3D voxel structures is difficult due to the cubic memory scaling of voxel structures and moreover the significant class imbalance caused by sparsity. We introduce Scaffold Diffusion, a generative model designed for sparse multi-category 3D voxel structures. By treating voxels as tokens, Scaffold Diffusion uses a discrete diffusion language model to generate 3D voxel structures. We show that discrete diffusion language models can be extended beyond inherently sequential domains such as text to generate spatially coherent 3D structures. We evaluate on Minecraft house structures from the 3D-Craft dataset and demonstrate that, unlike prior baselines and an auto-regressive formulation, Scaffold Diffusion produces realistic and coherent structures even when trained on data with over 98% sparsity. We provide an interactive viewer where readers can visualize generated samples and the generation process: https://scaffold.deepexploration.org/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。