无需真实代码参考，实时评估流程图转代码质量。
An Online Reference-Free Evaluation Framework for Flowchart Image-to-Code Generation
- 用OCR提取图像文字作伪参考，评估生成内容覆盖率。
- 通过视觉蕴含检测幻觉元素，精准识别错误生成部分。
- 适合在生产环境持续监控模型输出，无需人工标注。
视觉语言模型（VLM）正被广泛用于将流程图图像转换为结构化代码（如Mermaid）。在实际应用中，系统需处理无对应真实代码的任意输入，导致输出质量难以评估。本文提出一种无需参考的在线评估框架，仅凭输入图像和生成代码即可在推理时监测质量。框架引入两个自动化指标：$ ext{Recall}{ ext{OCR}}$ 通过OCR提取图像文本作为代理参考，估计内容覆盖程度；$ ext{Precision}{ ext{VE}}$ 利用视觉蕴含检测生成中的幻觉元素。二者调和平均得 $ ext{F1}{ ext{OCR-VE}}$，提供统一质量评分。在FlowVQA数据集上的验证显示，该框架与真实标签指标高度一致（平均皮尔逊相关系数分别为0.97、0.91和0.94），证实其作为生产环境中持续质量监控的可靠替代方案。
原文摘要 · Abstract (English)
Vision-Language Models (VLMs) are increasingly used in document processing pipelines to convert flowchart images into structured code (e.g., Mermaid). In production, these systems process arbitrary inputs for which no ground-truth code exists, making output quality difficult to assess. We propose a reference-free evaluation framework that monitors flowchart image-to-code generation quality at inference time, using only the input image and the generated output. The framework introduces two automated metrics: $\text{Recall}{\text{OCR}}$, which estimates content coverage by extracting text from the input image via OCR as a proxy reference, and $\text{Precision}{\text{VE}}$, which detects hallucinated elements through Visual Entailment against the original image. Their harmonic mean, $\text{F1}{\text{OCR-VE}}$, provides a unified quality score. Validation on the FlowVQA dataset shows strong agreement with ground-truth metrics (average Pearson's $r = 0.97$, $0.91$, and $0.94$ for Recall, Precision, and F1, respectively), confirming the framework's reliability as a practical, reference-free alternative for continuous quality monitoring in production settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。