用自然语言生成可物理组装的积木指令,支持超3000部件复杂结构。
Prompt-to-Parts: Generative AI for Physical Assembly and Scalable Instructions
- 基于积木词汇表与几何约束,将自然语言转为可建造步骤。
- 在卫星、飞机等复杂结构上生成有效组装序列,支持超3000部件。
- 适合制造与工程原型设计,为语言驱动制造提供新范式。
我们提出一种从自然语言描述生成物理可实现装配指令的框架。与无约束的文本到3D方法不同,该方法在离散零件词汇表内运行,确保几何合理性、连接约束和可建造顺序。利用LDraw作为富含文本的中间表示,我们证明大型语言模型可通过工具引导,生成包含超过3000个零件的积木原型的可执行分步构建序列与装配说明。我们引入一个用于程序化模型生成的Python库,并在复杂卫星、飞机及建筑领域评估了可建造输出。该方法旨在实现可演示的可扩展性、模块化和保真度,弥合语义设计意图与可制造输出之间的差距。物理原型可直接从自然语言规范生成。本工作提出一种新型基础语言,是此前基于像素的扩散方法或计算机辅助设计(CAD)模型所缺失的关键环节——无法支持复杂装配指令或组件互换。在四个原创设计中,这一“积木袋”方法充当物理接口:通过约束词汇将精确定位的积木位置与“词语库”连接,使任意功能需求编译为实体现实。这种一致且可重复的AI表征为设计开辟新可能,同时指导制造业与工程原型中的自然语言实现。
原文摘要 · Abstract (English)
We present a framework for generating physically realizable assembly instructions from natural language descriptions. Unlike unconstrained text-to-3D approaches, our method operates within a discrete parts vocabulary, enforcing geometric validity, connection constraints, and buildability ordering. Using LDraw as a text-rich intermediate representation, we demonstrate that large language models can be guided with tools to produce valid step-by-step construction sequences and assembly instructions for brick-based prototypes of more than 3000 assembly parts. We introduce a Python library for programmatic model generation and evaluate buildable outputs on complex satellites, aircraft, and architectural domains. The approach aims for demonstrable scalability, modularity, and fidelity that bridges the gap between semantic design intent and manufacturable output. Physical prototyping follows from natural language specifications. The work proposes a novel elemental lingua franca as a key missing piece from the previous pixel-based diffusion methods or computer-aided design (CAD) models that fail to support complex assembly instructions or component exchange. Across four original designs, this novel "bag of bricks" method thus functions as a physical API: a constrained vocabulary connecting precisely oriented brick locations to a "bag of words" through which arbitrary functional requirements compile into material reality. Given such a consistent and repeatable AI representation opens new design options while guiding natural language implementations in manufacturing and engineering prototyping.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。