用视觉模型精准识别电气图中表格与电路的关联关系。
ViRED: Prediction of Visual Relations in Engineering Drawings
- 基于视觉编码器+对象编码器+关系解码器架构，端到端建模图文关联
- 在工程图数据集上达到96%关系预测准确率，显著优于现有方法
- 适用于含大量元件的复杂电气图纸，推理速度快，适合工业场景
为准确理解工程图纸，需建立图像与图中描述表格之间的对应关系。现有文档理解方法多以文本为主，不适用于包含大量图像信息的文档。在视觉关系检测领域，任务结构本身限制了对所有实体对关系的评估能力。为此，我们提出一种基于视觉的关系检测模型ViRED，用于识别电气工程图中表格与电路之间的关联。模型由视觉编码器、对象编码器和关系解码器三部分构成。使用PyTorch实现并进行实验验证。结果表明，在工程图数据集上，该方法在关系预测任务中达到96%的准确率，显著优于现有方法；同时，即使在单张图包含大量对象的情况下，仍能保持快速推理速度。
原文摘要 · Abstract (English)
To accurately understand engineering drawings, it is essential to establish the correspondence between images and their description tables within the drawings. Existing document understanding methods predominantly focus on text as the main modality, which is not suitable for documents containing substantial image information. In the field of visual relation detection, the structure of the task inherently limits its capacity to assess relationships among all entity pairs in the drawings. To address this issue, we propose a vision-based relation detection model, named ViRED, to identify the associations between tables and circuits in electrical engineering drawings. Our model mainly consists of three parts: a vision encoder, an object encoder, and a relation decoder. We implement ViRED using PyTorch to evaluate its performance. To validate the efficacy of ViRED, we conduct a series of experiments. The experimental results indicate that, within the engineering drawing dataset, our approach attained an accuracy of 96\% in the task of relation prediction, marking a substantial improvement over existing methodologies. The results also show that ViRED can inference at a fast speed even when there are numerous objects in a single engineering drawing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。