用视觉语言模型从2D图纸自动生成3D参数化模型
From 2D CAD Drawings to 3D Parametric Models: A Vision-Language Approach
- 将2D图纸当图像输入，用ViT编码，不依赖矢量格式
- 自回归生成描述3D模型的通用文本，支持任意几何与属性
- 在柜子数据集上表现优于传统方法，灵活易扩展
本文提出CAD2Program，一种从2D CAD图纸重建3D参数化模型的新方法。受视觉语言模型成功启发，该方法不依赖特定任务的数据表示或算法。输入端，将2D CAD图纸视为光栅图像，使用标准ViT模型进行编码，无需关心原始格式，性能可媲美基于矢量图形的方法，且限制更少。输出端，通过自回归方式生成描述3D参数化模型的通用文本，相比使用固定尺寸槽位的领域专用序列表示，该文本表示更具灵活性，可轻松扩展至任意几何实体及语义或功能属性。在大规模柜子模型数据集上的实验表明，该方法有效可靠。
原文摘要 · Abstract (English)
In this paper, we present CAD2Program, a new method for reconstructing 3D parametric models from 2D CAD drawings. Our proposed method is inspired by recent successes in vision-language models (VLMs), and departs from traditional methods which rely on task-specific data representations and/or algorithms. Specifically, on the input side, we simply treat the 2D CAD drawing as a raster image, regardless of its original format, and encode the image with a standard ViT model. We show that such an encoding scheme achieves competitive performance against existing methods that operate on vector-graphics inputs, while imposing substantially fewer restrictions on the 2D drawings. On the output side, our method auto-regressively predicts a general-purpose language describing 3D parametric models in text form. Compared to other sequence modeling methods for CAD which use domain-specific sequence representations with fixed-size slots, our text-based representation is more flexible, and can be easily extended to arbitrary geometric entities and semantic or functional properties. Experimental results on a large-scale dataset of cabinet models demonstrate the effectiveness of our method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。