用数亿方块训练3D生成模型，可交互编辑虚拟世界。
Dream-Cubed: Controllable Generative Modeling in Minecraft by Training on Billions of Cubes
- 以方块为基本单元，直接在区块空间生成3D环境。
- 在数十亿令牌数据上训练，支持交互式补全与扩展。
- 开源数据集和模型，适合3D生成与游戏开发研究。
我们提出Dream-Cubed，一个大规模Minecraft世界体素级数据集，以及一套以方块为组合单元的生成模型家族，用于高效生成可交互的3D环境。该数据集包含数十亿令牌，由程序化生物群系地形与高质量人工创作地图精心混合构成。我们首次对体素生成中的3D扩散模型进行了大规模研究，分析了离散与连续扩散形式、数据构成及架构设计选择。模型直接在方块空间操作，实现高效且语义合理的生成，支持用户通过已构建区块进行补绘与扩画等交互工作流。为定量评估，我们改编FID指标以衡量真实与生成世界渲染间的语义差异，并通过人类偏好实验验证生成质量。我们公开发布完整数据集、代码及所有预训练模型，旨在为未来高效生成结构化、可交互3D环境的研究奠定基础。
原文摘要 · Abstract (English)
We introduce Dream-Cubed, a large-scale dataset of Minecraft worlds at voxel resolution, and a family of models using cubes as powerful compositional units for efficient generation of interactive 3D environments. Dream-Cubed comprises tens of billions of tokens from a carefully curated mixture of procedural biome terrain and high-quality human-authored maps. We use this dataset to conduct the first large-scale study of 3D diffusion models for voxel generation, analyzing discrete and continuous diffusion formulations, data compositions, and architectural design choices. Our models operate directly in the space of blocks, enabling efficient and semantically grounded generation while supporting interactive user workflows such as inpainting and outpainting from user-authored blocks. To quantitatively evaluate our models, we adapt the FID metric to assess semantic differences between real and generated world renderings, and validate generation quality through a human preference study. We release the full dataset, code, and all our pretrained models, which we hope will provide a foundation for future research in efficient generative modeling for structured, interactive 3D environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。