用自然语言修改已有CAD模型,自动合成训练数据并分步完成定位与填充。
CAD-Editor: A Locate-then-Infill Framework with Automated Training Data Synthesis for Text-Based CAD Editing
- 先定位需修改区域,再填入新设计,分两步实现精准编辑。
- 自动生成带对应关系的三元组数据,解决标注难问题。
- 适合工业设计、自动化建模人员快速迭代模型。
计算机辅助设计(CAD)在各行业不可或缺。基于文本的CAD编辑能根据自然语言指令自动修改CAD模型,潜力巨大但研究尚少。现有方法多聚焦于设计变体生成或文本驱动的CAD生成,或缺乏文本控制能力,或忽略已有模型的约束。本文提出首个基于文本的CAD编辑框架CAD-Editor。为解决训练所需精确对应三元组数据稀缺的问题,我们设计了自动化数据合成流程:利用设计变体模型生成原始与修改后的CAD对,并通过大视觉语言模型(LVLMs)提炼差异生成编辑指令。针对文本编辑任务的复合性,提出“定位-填入”框架,将任务分解为两个子任务:定位需修改区域,以及在该区域填入合适的新结构。大型语言模型(LLMs)作为两大子任务的核心,依托其自然语言理解与CAD知识能力。实验表明,CAD-Editor在定量和定性指标上均表现优异。代码已开源:https://github.com/microsoft/CAD-Editor。
原文摘要 · Abstract (English)
Computer Aided Design (CAD) is indispensable across various industries. \emph{Text-based CAD editing}, which automates the modification of CAD models based on textual instructions, holds great potential but remains underexplored. Existing methods primarily focus on design variation generation or text-based CAD generation, either lacking support for text-based control or neglecting existing CAD models as constraints. We introduce \emph{CAD-Editor}, the first framework for text-based CAD editing. To address the challenge of demanding triplet data with accurate correspondence for training, we propose an automated data synthesis pipeline. This pipeline utilizes design variation models to generate pairs of original and edited CAD models and employs Large Vision-Language Models (LVLMs) to summarize their differences into editing instructions. To tackle the composite nature of text-based CAD editing, we propose a locate-then-infill framework that decomposes the task into two focused sub-tasks: locating regions requiring modification and infilling these regions with appropriate edits. Large Language Models (LLMs) serve as the backbone for both sub-tasks, leveraging their capabilities in natural language understanding and CAD knowledge. Experiments show that CAD-Editor achieves superior performance both quantitatively and qualitatively. The code is available at \url {https://github.com/microsoft/CAD-Editor}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。