通过编码箭头方向提升视觉语言模型对流程图的理解能力
Arrow-Guided VLM: Enhancing Flowchart Understanding via Arrow Direction Encoding
- 显式编码箭头方向，构建结构化提示引导模型推理
- 整体准确率从80%提升至89%，下一步预测准确率达100%
- 适合需要精确理解流程逻辑的开发者与业务分析人员
流程图在软件设计和业务流程分析中至关重要，但现有视觉语言模型（VLM）常误解其箭头方向与拓扑结构。本文提出七阶段流水线，分为三部分：(1) 识别节点与箭头端点；(2) 通过光学字符识别（OCR）提取节点文本；(3) 构建结构化提示引导VLM。在由30个标注流程图提炼出的90题基准上测试，方法将整体准确率从80%提升至89%（+9个百分点），无任务微调。下一步查询准确率从25/30升至30/30（100%，+17个百分点）；分支结果类问题改善有限，前步查询仍困难。通过大语言模型作为裁判的平行评估验证了相同趋势，证实显式箭头编码的优势。局限包括依赖检测器与OCR精度、评测集较小，以及多输入边节点残余错误。未来工作将扩充合成与手写流程图数据集，并评估在业务流程建模与符号（BPMN）及统一建模语言（UML）上的表现。
原文摘要 · Abstract (English)
Flowcharts are indispensable tools in software design and business-process analysis, yet current vision-language models (VLMs) frequently misinterpret the directional arrows and graph topology that set these diagrams apart from natural images. We introduce a seven-stage pipeline grouped into three broader processes: (1) arrow-aware detection of nodes and arrow endpoints; (2) optical character recognition (OCR) to extract node text; and (3) construction of a structured prompt that guides the VLMs. Tested on a 90-question benchmark distilled from 30 annotated flowcharts, the method raises overall accuracy from 80 % to 89 % (+9 percentage points) without any task-specific fine-tuning. The gain is most pronounced for next-step queries (25/30 -> 30/30; 100 %, +17 pp); branch-result questions improve more modestly, and before-step questions remain difficult. A parallel evaluation with an LLM-as-a-Judge protocol shows the same trends, reinforcing the advantage of explicit arrow encoding. Limitations include dependence on detector and OCR precision, the small evaluation set, and residual errors at nodes with multiple incoming edges. Future work will enlarge the benchmark with synthetic and handwritten flowcharts and assess the approach on Business Process Model and Notation (BPMN) and Unified Modeling Language (UML).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。