用普通照片生成精准的3D设计模型,无需专业设备
CADCrafter: Generating Computer-Aided Design Models from Unconstrained Images
- 通过合成数据训练,从任意照片还原参数化CAD模型
- 新几何编码器捕捉复杂形状,实测可处理未见过的物体
- 用自动校验反馈优化生成质量,适合非专业人士使用
从真实世界创建CAD数字孪生对制造、设计和仿真至关重要。但现有方法依赖昂贵的3D扫描与繁琐后处理。为实现用户友好的设计流程,我们探索从无约束真实世界CAD图像中逆向工程建模,这类图像可由各类用户轻松获取。然而,真实世界CAD数据稀缺,直接训练面临挑战。为此,我们提出CADCrafter——一个仅在合成无纹理CAD数据上训练,却能在真实图像上测试的图像到参数化CAD模型生成框架。为弥合图像与参数化CAD模型间的表示差异,引入几何编码器以准确捕捉多样几何特征。且该几何特征的无纹理特性有助于泛化至真实场景。由于将CAD参数序列编译为显式模型是非可微过程,网络训练缺乏显式几何监督。为此,我们采用直接偏好优化(DPO)方法,利用自动代码检查器反馈来微调模型,确保生成序列的几何有效性。此外,我们收集了一个真实世界数据集,包含多视角图像与对应CAD命令序列对,用于评估方法。实验表明,本方法能稳健处理真实无约束的CAD图像,并可泛化至未见过的一般物体。
原文摘要 · Abstract (English)
Creating CAD digital twins from the physical world is crucial for manufacturing, design, and simulation. However, current methods typically rely on costly 3D scanning with labor-intensive post-processing. To provide a user-friendly design process, we explore the problem of reverse engineering from unconstrained real-world CAD images that can be easily captured by users of all experiences. However, the scarcity of real-world CAD data poses challenges in directly training such models. To tackle these challenges, we propose CADCrafter, an image-to-parametric CAD model generation framework that trains solely on synthetic textureless CAD data while testing on real-world images. To bridge the significant representation disparity between images and parametric CAD models, we introduce a geometry encoder to accurately capture diverse geometric features. Moreover, the texture-invariant properties of the geometric features can also facilitate the generalization to real-world scenarios. Since compiling CAD parameter sequences into explicit CAD models is a non-differentiable process, the network training inherently lacks explicit geometric supervision. To impose geometric validity constraints, we employ direct preference optimization (DPO) to fine-tune our model with the automatic code checker feedback on CAD sequence quality. Furthermore, we collected a real-world dataset, comprised of multi-view images and corresponding CAD command sequence pairs, to evaluate our method. Experimental results demonstrate that our approach can robustly handle real unconstrained CAD images, and even generalize to unseen general objects.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。