arXiv:2510.17157cs.CVcs.AI2025-10被引 3

用单图生成更准更简的可编辑3D建模代码

GACO-CAD: Geometry-Augmented and Conciseness-Optimized CAD Model Generation from Single Image

  • 用深度图和法向图做几何先验,提升2D转3D的准确性
  • 通过奖励机制让生成的建模代码更短、少冗余
  • 适合想快速生成工业级参数化模型的人

从单张图像生成可编辑、参数化的CAD模型,有望降低工业概念设计门槛。但现有多模态大模型因空间推理能力有限,难以准确从2D图像推断3D几何。为此,提出GACO-CAD,一种两阶段后训练框架,旨在同时提升生成模型的几何精度与建模过程的简洁性。第一阶段监督微调中,将深度图和表面法向图作为密集几何先验,与RGB图像组成多通道输入,在单视角重建任务中提供互补的空间线索,增强模型对3D结构的恢复能力。第二阶段强化学习中,引入群体长度奖励,在保持高几何保真度的同时,鼓励生成更紧凑、少冗余的参数化建模序列。采用简单动态权重策略稳定训练。在DeepCAD和Fusion360数据集上的实验表明,使用相同MLLM骨干网络,GACO-CAD在代码有效性、几何精度和建模简洁性上均达到当前最优水平。

原文摘要 · Abstract (English)

Generating editable, parametric CAD models from a single image holds great potential to lower the barriers of industrial concept design. However, current multi-modal large language models (MLLMs) still struggle with accurately inferring 3D geometry from 2D images due to limited spatial reasoning capabilities. We address this limitation by introducing GACO-CAD, a novel two-stage post-training framework. It is designed to achieve a joint objective: simultaneously improving the geometric accuracy of the generated CAD models and encouraging the use of more concise modeling procedures. First, during supervised fine-tuning, we leverage depth and surface normal maps as dense geometric priors, combining them with the RGB image to form a multi-channel input. In the context of single-view reconstruction, these priors provide complementary spatial cues that help the MLLM more reliably recover 3D geometry from 2D observations. Second, during reinforcement learning, we introduce a group length reward that, while preserving high geometric fidelity, promotes the generation of more compact and less redundant parametric modeling sequences. A simple dynamic weighting strategy is adopted to stabilize training. Experiments on the DeepCAD and Fusion360 datasets show that GACO-CAD achieves state-of-the-art performance under the same MLLM backbone, consistently outperforming existing methods in terms of code validity, geometric accuracy, and modeling conciseness.

3D生成参数化建模多模态工业设计

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。