用AI把手绘草图转为可编辑的精确流程图
See it. Say it. Sorted: Agentic System for Compositional Diagram Generation
- 用视觉语言模型+大模型迭代生成，不需训练
- 在10个真实流程图草图上优于闭源顶尖模型
- 输出可编程SVG，适合演示工具集成
我们研究手绘草图到流程图的生成：将粗糙手绘转化为精确、组合式的图表。扩散模型擅长写实但缺乏空间精度、对齐和符号结构能力。我们提出See it. Say it. Sorted.，一个无需训练的智能体系统，结合视觉-语言模型（VLM）与大语言模型（LLM），生成可编辑的可缩放矢量图形（SVG）程序。系统运行迭代循环：批评者VLM提出少量定性关系修改；多个候选LLM以不同策略（保守→激进、替代、专注）合成SVG更新；裁判者VLM选择最优方案，确保稳定改进。该设计强调定性推理而非脆弱的数值估计，保持全局约束（如对齐、连接性），并自然支持人工干预。在10个来自已发表论文的流程图草图上，本方法比两个前沿闭源图像生成大模型（GPT-5和Gemini-2.5-Pro）更忠实还原布局与结构，准确组合元件（如多头箭头），且不插入多余文字。因输出为程序化SVG，可通过API轻松扩展至演示工具（如PowerPoint），并可借助优化提示与任务专用工具进行定制。代码已开源：https://github.com/hantaoZhangrichard/see_it_say_it_sorted.git。
原文摘要 · Abstract (English)
We study sketch-to-diagram generation: converting rough hand sketches into precise, compositional diagrams. Diffusion models excel at photorealism but struggle with the spatial precision, alignment, and symbolic structure required for flowcharts. We introduce See it. Say it. Sorted., a training-free agentic system that couples a Vision-Language Model (VLM) with Large Language Models (LLMs) to produce editable Scalable Vector Graphics (SVG) programs. The system runs an iterative loop in which a Critic VLM proposes a small set of qualitative, relational edits; multiple candidate LLMs synthesize SVG updates with diverse strategies (conservative->aggressive, alternative, focused); and a Judge VLM selects the best candidate, ensuring stable improvement. This design prioritizes qualitative reasoning over brittle numerical estimates, preserves global constraints (e.g., alignment, connectivity), and naturally supports human-in-the-loop corrections. On 10 sketches derived from flowcharts in published papers, our method more faithfully reconstructs layout and structure than two frontier closed-source image generation LLMs (GPT-5 and Gemini-2.5-Pro), accurately composing primitives (e.g., multi-headed arrows) without inserting unwanted text. Because outputs are programmatic SVGs, the approach is readily extensible to presentation tools (e.g., PowerPoint) via APIs and can be specialized with improved prompts and task-specific tools. The codebase is open-sourced at https://github.com/hantaoZhangrichard/see_it_say_it_sorted.git.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。