用视觉语言模型直接从BPMN图片提取结构化数据，无需原始文件。
Structured Extraction from Business Process Diagrams Using Vision-Language Models
- 基于视觉语言模型，从BPMN图像中直接提取结构化信息。
- 结合OCR提升文本识别，使元素提取准确率显著提高。
- 适用于无源文件时的流程图分析，适合流程自动化研究者。
业务流程建模与标注（BPMN）是表示复杂业务流程的通用标准。尽管BPMN图常以图像形式交换，现有方法主要依赖XML表示进行计算分析。本文提出一种利用视觉语言模型（VLMs）直接从图像中提取BPMN图结构化JSON表示的流程，无需源模型文件或文本注释。我们结合光学字符识别（OCR）进行文本增强，并将生成的元素列表与源自原始XML文件的真值数据对比评估。该方法在无原始文件场景下仍能实现稳健的组件提取。我们对多个VLM进行了基准测试，发现使用OCR进行文本增强后，多个模型性能提升。此外，我们还开展了广泛的统计分析和提示词消融实验，深入理解了不同方法对模型表现的影响。
原文摘要 · Abstract (English)
Business Process Model and Notation (BPMN) is a widely adopted standard for representing complex business workflows. While BPMN diagrams are often exchanged as visual images, existing methods primarily rely on XML representations for computational analysis. In this work, we present a pipeline that leverages Vision-Language Models (VLMs) to extract structured JSON representations of BPMN diagrams directly from images, without requiring source model files or textual annotations. We also incorporate optical character recognition (OCR) for textual enrichment and evaluate the generated element lists against ground truth data derived from the source XML files. Our approach enables robust component extraction in scenarios where original source files are unavailable. We benchmark multiple VLMs and observe performance improvements in several models when OCR is used for text enrichment. In addition, we conducted extensive statistical analyses of OCR-based enrichment methods and prompt ablation studies, providing a clearer understanding of their impact on model performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。