让AI用自然语言读CAD图纸，精准提取零件信息
QueryCAD: Grounded Question Answering for CAD Models
- 用自然语言描述定位零件，基于开放词汇分割模型识别
- 构建首个CAD问答基准，支持模型性能评估
- 已集成至机器人程序自动生成框架，适合工业自动化研究者
CAD模型在工业中广泛应用，对机器人自动化至关重要，但现有AI方法极少利用它们进行分析或信息提取。为此，我们提出QueryCAD，首个针对CAD模型的问答系统，支持通过自然语言查询精确提取信息。QueryCAD引入SegCAD——一种我们开发的开放词汇实例分割模型，可根据零件描述精准识别和选择目标部件。我们还构建了首个CAD问答评测基准，为后续研究奠定基础。最后，将QueryCAD集成至自动机器人程序生成框架中，验证其提升深度学习机器人解决方案的能力，使模型可直接处理CAD模型。项目主页：https://claudius-kienle.github.com/querycad。
原文摘要 · Abstract (English)
CAD models are widely used in industry and are essential for robotic automation processes. However, these models are rarely considered in novel AI-based approaches, such as the automatic synthesis of robot programs, as there are no readily available methods that would allow CAD models to be incorporated for the analysis, interpretation, or extraction of information. To address these limitations, we propose QueryCAD, the first system designed for CAD question answering, enabling the extraction of precise information from CAD models using natural language queries. QueryCAD incorporates SegCAD, an open-vocabulary instance segmentation model we developed to identify and select specific parts of the CAD model based on part descriptions. We further propose a CAD question answering benchmark to evaluate QueryCAD and establish a foundation for future research. Lastly, we integrate QueryCAD within an automatic robot program synthesis framework, validating its ability to enhance deep-learning solutions for robotics by enabling them to process CAD models (https://claudius-kienle.github.com/querycad).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。