用2D手绘草图生成稳定3D结构，无需专业工具
"Stack It Up!": 3D Stable Structure Generation from 2D Hand-drawn Sketch
- 用抽象关系图捕捉草图中的几何与稳定性模式
- 生成多层稳定3D结构，比基线更像原图且更稳固
- 适合非专业人士快速设计复杂积木式结构
想象一个孩子画出埃菲尔铁塔的草图，并让机器人将其变为实物。当前的机器人操作系统无法直接处理此类草图——它们需要精确的3D积木位姿作为目标，而这又依赖于结构分析和专业工具如CAD。我们提出StackItUp，一种使非专业人士仅通过2D正视手绘草图即可指定复杂3D结构的系统。StackItUp引入抽象关系图，弥合粗糙草图与精确3D积木布局之间的鸿沟，捕捉符号化几何关系（如“左-右”）和稳定性模式（如“双柱桥”），同时忽略草图中的噪声度量细节。随后利用组合扩散模型将该图映射为3D位姿，并通过迭代预测隐藏的内部及背面支撑结构——这些对稳定性至关重要但未在草图中体现。在著名地标与现代房屋设计草图上的评估显示，StackItUp持续生成稳定、多层的3D结构，且在稳定性和视觉相似性上均优于所有基线方法。
原文摘要 · Abstract (English)
Imagine a child sketching the Eiffel Tower and asking a robot to bring it to life. Today's robot manipulation systems can't act on such sketches directly-they require precise 3D block poses as goals, which in turn demand structural analysis and expert tools like CAD. We present StackItUp, a system that enables non-experts to specify complex 3D structures using only 2D front-view hand-drawn sketches. StackItUp introduces an abstract relation graph to bridge the gap between rough sketches and accurate 3D block arrangements, capturing the symbolic geometric relations (e.g., left-of) and stability patterns (e.g., two-pillar-bridge) while discarding noisy metric details from sketches. It then grounds this graph to 3D poses using compositional diffusion models and iteratively updates it by predicting hidden internal and rear supports-critical for stability but absent from the sketch. Evaluated on sketches of iconic landmarks and modern house designs, StackItUp consistently produces stable, multilevel 3D structures and outperforms all baselines in both stability and visual resemblance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。