用文字生成可搭建且物理稳定的积木结构，支持人工和机器人组装。
Generating Physically Stable and Buildable Brick Structures from Text
- 基于文本提示自回归生成积木序列，结合物理约束动态修正不可行方案。
- 在47,000个结构数据集上训练，生成结果稳定、多样且贴合文本描述。
- 支持彩色纹理生成，成果可由人或机械臂实际搭建，适合创意设计与教育场景。
我们提出BrickGPT，首个从文本提示生成物理稳定且可连接积木结构的方法。为此，构建了一个大规模物理稳定积木结构数据集，包含超过47,000个结构，对应28,000个独特3D物体及详细描述，并训练自回归大语言模型通过下一词预测逐步生成积木。为提升稳定性，在自回归推理中引入高效的可行性检测与物理感知回滚机制，利用物理法则与装配约束剔除不可行的积木预测。实验表明，BrickGPT能生成稳定、多样且美观的积木结构，与输入文本高度一致。我们还开发了基于文本的积木着色方法，生成带颜色与纹理的设计。结果显示，这些设计可由人类手工搭建，也可由机器人手臂自动组装。我们已公开数据集StableText2Brick、代码与模型，详见项目主页：https://avalovelace1.github.io/BrickGPT/
原文摘要 · Abstract (English)
We introduce BrickGPT, the first approach for generating physically stable interconnecting brick assembly models from text prompts. To achieve this, we construct a large-scale, physically stable dataset of brick structures, along with their associated captions, and train an autoregressive large language model to predict the next brick to add via next-token prediction. To improve the stability of the resulting designs, we employ an efficient validity check and physics-aware rollback during autoregressive inference, which prunes infeasible token predictions using physics laws and assembly constraints. Our experiments show that BrickGPT produces stable, diverse, and aesthetically pleasing brick structures that align closely with the input text prompts. We also develop a text-based brick texturing method to generate colored and textured designs. We show that our designs can be assembled manually by humans and automatically by robotic arms. We release our new dataset, StableText2Brick, containing over 47,000 brick structures of over 28,000 unique 3D objects accompanied by detailed captions, along with our code and models at the project website: https://avalovelace1.github.io/BrickGPT/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。