将流程图转为可执行代码，让机器读懂编程文档中的图形。
Towards Making Flowchart Images Machine Interpretable
- 用Transformer模型将流程图解析为语法正确的Python代码。
- 在11,884张流程图上训练，生成准确率显著优于基线方法。
- 适合需要自动化提取算法逻辑的开发者或文档处理系统。
计算机编程教材和软件文档中常包含流程图以展示算法流程，但现代OCR引擎通常将这些流程图识别为图像并忽略其内容。本文旨在使流程图图像对机器可读，通过将其转换为可执行的Python代码。受自然语言到代码生成研究的启发，我们提出一种基于Transformer的框架FloCo-T5。该模型能有效学习编程语言的语义、结构与模式，从而生成语法正确的代码。我们还设计了一种任务特定的预训练目标，使用大量保持逻辑一致性的增强代码样本来预训练模型。为进一步严谨评估该问题，我们构建了FloCo数据集，包含11,884张流程图及其对应的Python代码。实验表明，FloCo-T5在代码生成指标上明显优于相关基线方法。我们已公开数据集与实现代码。
原文摘要 · Abstract (English)
Computer programming textbooks and software documentations often contain flowcharts to illustrate the flow of an algorithm or procedure. Modern OCR engines often tag these flowcharts as graphics and ignore them in further processing. In this paper, we work towards making flowchart images machine-interpretable by converting them to executable Python codes. To this end, inspired by the recent success in natural language to code generation literature, we present a novel transformer-based framework, namely FloCo-T5. Our model is well-suited for this task,as it can effectively learn semantics, structure, and patterns of programming languages, which it leverages to generate syntactically correct code. We also used a task-specific pre-training objective to pre-train FloCo-T5 using a large number of logic-preserving augmented code samples. Further, to perform a rigorous study of this problem, we introduce theFloCo dataset that contains 11,884 flowchart images and their corresponding Python codes. Our experiments show promising results, and FloCo-T5 clearly outperforms related competitive baselines on code generation metrics. We make our dataset and implementation publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。