用改进的Transformer模型自动解析工程图,提取结构化信息。
Automated Parsing of Engineering Drawings for Structured Information Extraction Using a Fine-tuned Document Understanding Transformer
- 结合定向框检测与Transformer,实现图文联合解析
- 单模型整体表现优于分项模型,GD&T精度达94.77%
- 适合智能制造、工业自动化等需高精度图纸处理场景
从二维工程图中准确提取关键信息对高精度制造至关重要。人工提取效率低,传统OCR难以应对复杂布局与重叠符号,输出常为非结构化数据。本文提出一种混合深度学习框架,将定向边界框(OBB)检测模型与基于Transformer的文档解析模型(Donut)结合。使用自建标注数据集训练YOLOv11,识别九类关键元素:几何尺寸与公差(GD&T)、一般公差、尺寸、材料、注释、圆角、表面粗糙度、螺纹和标题栏。检测出的OBB被裁剪并标注,用于微调Donut生成结构化JSON。微调策略包括全类别共享模型与类别专用模型。结果表明,共享模型在各项指标上均优于专用模型,GD&T精度达94.77%,多数类别召回率达100%,F1分数达97.3%,幻觉率仅5.23%。该框架显著提升准确性,减少人工成本,支持精密制造领域的规模化应用。
原文摘要 · Abstract (English)
Accurate extraction of key information from 2D engineering drawings is crucial for high-precision manufacturing. Manual extraction is slow and labor-intensive, while traditional Optical Character Recognition (OCR) techniques often struggle with complex layouts and overlapping symbols, resulting in unstructured outputs. To address these challenges, this paper proposes a novel hybrid deep learning framework for structured information extraction by integrating an Oriented Bounding Box (OBB) detection model with a transformer-based document parsing model (Donut). An in-house annotated dataset is used to train YOLOv11 for detecting nine key categories: Geometric Dimensioning and Tolerancing (GD&T), General Tolerances, Measures, Materials, Notes, Radii, Surface Roughness, Threads, and Title Blocks. Detected OBBs are cropped into images and labeled to fine-tune Donut for structured JSON output. Fine-tuning strategies include a single model trained across all categories and category-specific models. Results show that the single model consistently outperforms category-specific ones across all evaluation metrics, achieving higher precision (94.77% for GD&T), recall (100% for most categories), and F1 score (97.3%), while reducing hallucinations (5.23%). The proposed framework improves accuracy, reduces manual effort, and supports scalable deployment in precision-driven industries.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。