arXiv:2605.30794cs.CVcs.AI2026-05中稿 · iclm2026, add gith…被引 2

首个机械图理解基准数据集,提升大模型对工程图的识图与推理能力。

MechVQA: Benchmarking and Enhancing Multimodal LLMs on Comprehensive Mechanical Drawing Understanding

论文配图:MechVQA: Benchmarking and Enhancing Multimodal LLMs on Comprehensive Mechanical Drawing Understanding
图 1 · 摘自论文原文
  • 构建半自动数据管道生成3.3千张高密度机械图与2.1万问答对
  • 新模型MechVL在基准上比最强闭源模型高7.57个百分点
  • 覆盖识别、推理、判断三类任务,适合工业设计与质检场景

多模态大语言模型在通用视觉问答中表现优异,但在机械工程图上仍显脆弱,因标注密度高、领域知识弱,且受严格投影规则和几何约束影响,空间关系推理不可靠,易遗漏关键线索导致错误。为此,我们提出首个全面的机械图理解数据集MechVQA,通过半自动化构建与质量控制流程,包含3.3k张高密度图像和2.1万个问答对,涵盖10个细粒度任务,分属识别、推理、判断三个能力层级,为评估和提升MLLM在真实机械图上的理解能力提供测试平台。在此基础上,我们采用多阶段训练范式开发了MechVL模型,建立强领域专用基线。实验表明,MechVL在MechVQA总分上优于最强闭源基线7.57个百分点,显著提升机械图理解能力,可为机械设计与检测场景中的MLLM部署提供可复用基础。

原文摘要 · Abstract (English)

Multimodal Large Language Models (MLLMs) have demonstrated significant achievements in general visual question answering (VQA) tasks. However, they remain brittle on mechanical engineering drawings, where high annotation density and weak domain knowledge, compounded by unreliable spatial relation reasoning under strict projection rules and geometric constraints, make decisive cues easy to miss and frequently lead to wrong answers. To bridge this gap, we introduce the first comprehensive mechanical drawing understanding dataset, MechVQA, created through a semi-automated construction and quality-control pipeline. MechVQA contains 3.3k high-density pictures with 21K question-answer pairs, spanning 10 different fine-grained tasks across three capability levels: Recognition, Reasoning, and Judging, providing a testbed to evaluate and improve MLLM understanding on real-world mechanical drawings. On top of MechVQA, we then develop the MechVL model through a multi-stage training paradigm, building a strong domain-specialized baseline. Extensive experimental results demonstrate that MechVL outperforms the strongest closed-source baseline by 7.57 percentage points on the MechVQA total score, significantly enhancing mechanical drawing understanding ability and providing a reusable foundation for deploying MLLMs in mechanical design and inspection scenarios.

机械图理解多模态大模型基准数据集工程应用

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。