构建大规模3D装配推理基准,评估模型对工业机械的物理合理性理解能力。
OmniCAD: A Large-Scale Benchmark for 3D Spatial Reasoning in Robotics Assemblies

- 设计包含2.5万个机械装配体的多领域基准,每装配体平均12个零件、21种连接关系。
- 测试发现现有视觉语言模型在复杂装配中常出现位置错误、连接无效和零件穿插问题。
- 支持工具增强的智能体推理,适合研究机器人装配、工业认知与具身智能的学者。
近期视觉语言模型(VLMs)在机器人感知与空间推理方面展现出强大能力,但其对复杂机械装配的推理能力仍待深入探索。我们提出OmniCAD,一个覆盖多种工业系统的装配感知3D空间推理大规模基准,涵盖机器人机构、汽车部件、航空航天结构及农业机械。OmniCAD包含2.5万个机械装配体,平均每个装配体有12个零件,21种类型的关系连接。每个装配体均配有经人工验证的3D模型及20个视角的渲染图。该基准评估三项能力:(1) 零件级3D空间推理,要求预测零件位置与朝向;(2) 零件间关系推理,要求识别配合关系与装配约束;(3) 工具增强的智能体推理,模型可迭代选择视角、检视视觉证据并修正预测。实验表明,当前VLMs在工业装配推理中表现不佳,常出现姿态错误、无效连接、零件穿插,且随装配复杂度提升性能显著下降。我们将开源该基准、评估代码与工具接口,以推动准确、物理合理且可扩展的3D装配推理研究。
原文摘要 · Abstract (English)
Recent vision-language models (VLMs) show strong capabilities in robotic perception and spatial reasoning, yet their ability to reason about complex mechanical assemblies remains underexplored. We introduce OmniCAD, a large-scale benchmark for assembly-aware 3D spatial reasoning across diverse industrial systems, including robotic mechanisms, automotive components, aerospace structures, and agricultural machinery. OmniCAD contains 25k mechanical assemblies, with an average of 12 parts per assembly and 21 types of mate relationships. Each assembly includes a human-verified ground-truth 3D model and renderings from 20 viewpoints. The benchmark evaluates three capabilities: (1) component-level 3D spatial reasoning, requiring prediction of part positions and orientations; (2) part-to-part relational reasoning, requiring identification of mating relationships and assembly constraints; and (3) tool-augmented agentic reasoning, where models iteratively select viewpoints, inspect visual evidence, and refine predictions. Experiments show that current VLMs struggle with industrial assembly reasoning, often producing inaccurate poses, invalid mating relationships, part interpenetration, and degraded performance as assembly complexity increases. We will open-source the benchmark, evaluation code, and tool interfaces to support research on accurate, physically valid, and scalable 3D assembly reasoning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。