构建工业级程序化CAD生成评估基准,揭示当前模型在参数化设计上的严重缺陷。
BenchCAD: A Comprehensive, Industry-Standard Benchmark for Programmatic CAD

- 构建跨106类工业零件的17900个可执行代码基准,涵盖视觉、文本多模态输入
- 实测显示主流模型仅能还原粗略外形,90%以上存在结构缺失或操作误用
- 适合关注工业自动化、参数化建模与多模态生成落地的研究者和工程师
工业级计算机辅助设计(CAD)代码生成要求模型从视觉或文本输入中生成可执行的参数化程序。该任务不仅需识别零件外轮廓,还需理解其三维结构、推断工程参数,并选择符合实际设计制造流程的CAD操作。尽管多模态大语言模型(MLLMs)在此领域前景广阔,但极少在真实工业场景中联合评估其综合能力。本文提出BenchCAD,一个统一的工业级CAD推理评估基准。BenchCAD包含跨106类工业零件家族的17,900个执行验证的CadQuery程序,涵盖锥齿轮、压缩弹簧、螺旋钻等可复用工程设计。评估涵盖视觉问答、代码问答、图像到代码生成及指令引导的代码编辑,支持对感知、参数抽象与可执行程序合成的细粒度分析。在10余种前沿模型上测试发现,当前系统虽能恢复粗略外廓,但难以生成忠实的参数化CAD程序,常见失败包括遗漏精细3D结构、误解工业设计参数,以及将扫掠、放样、扭挤等关键操作替换为简单的草图加拉伸模式。微调与强化学习可提升分布内性能,但对未见零件家族的泛化能力仍有限。结果表明,BenchCAD可作为衡量与提升多模态CAD自动化工业适配性的基准。
原文摘要 · Abstract (English)
Industrial Computer-Aided Design (CAD) code generation requires models to produce executable parametric programs from visual or textual inputs. Beyond recognizing the outer shape of a part, this task involves understanding its 3D structure, inferring engineering parameters, and choosing CAD operations that reflect how the part would be designed and manufactured. Despite the promise of Multimodal large language models (MLLMs) for this task, they are rarely evaluated on whether these capabilities jointly hold in realistic industrial CAD settings. We present BenchCAD, a unified benchmark for industrial CAD reasoning. BenchCAD contains 17,900 execution-verified CadQuery programs across 106 industrial part families, including bevel gears, compression springs, twist drills, and other reusable engineering designs. It evaluates models through visual question answering, code question answering, image-to-code generation, and instruction-guided code editing, enabling fine-grained analysis across perception, parametric abstraction, and executable program synthesis. Across 10+ frontier models, BenchCAD shows that current systems often recover coarse outer geometry but fail to produce faithful parametric CAD programs. Common failures include missing fine 3D structure, misinterpreting industrial design parameters, and replacing essential operations such as sweeps, lofts, and twist-extrudes with simpler sketch-and-extrude patterns. Fine-tuning and reinforcement learning improve in-distribution performance, but generalization to unseen part families remains limited. These results position BenchCAD as a benchmark for measuring and improving the industrial readiness of multimodal CAD automation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。