让科学插图可编辑,用多智能体生成矢量图
From Pixels to Paths: A Multi-Agent Framework for Editable Scientific Illustration
- 用三个角色智能体协同生成可编辑的矢量图
- 支持元素级修改,插图可后期任意调整
- 提供七维评估体系,适合科研绘图人员
科学插图需要高信息密度且支持后期编辑。现有生成模型存在两大缺陷:一是图像生成模型输出位图,缺乏语义结构,无法访问或修改独立视觉元素;二是基于代码的生成方法(如TikZ或SVG)虽有元素级控制,但需经历“编写-编译-评审”的繁琐循环,操作不直观。为此,我们提出VisPainter,一个基于模型上下文协议的多智能体框架,由管理器、设计师和工具箱三个模块协同工作,生成兼容标准矢量图形软件的图表。该模块化角色设计使每个元素显式表达并可独立操控,实现真正的元素级控制,支持后续任意添加与修改。为系统评估科学插图质量,我们构建了VisBench基准,包含七维评价指标,从内容、布局、视觉感知和交互成本四个维度评估高信息密度插图。通过大量消融实验验证了架构合理性与评估方法可靠性。最终对多种视觉语言模型进行评估,给出公平可信的性能排名,并量化分析角色分工、步骤控制与描述对插图质量的影响。
原文摘要 · Abstract (English)
Scientific illustrations demand both high information density and post-editability. However, current generative models have two major limitations: Frist, image generation models output rasterized images lacking semantic structure, making it impossible to access, edit, or rearrange independent visual components in the images. Second, code-based generation methods (TikZ or SVG), although providing element-level control, force users into the cumbersome cycle of "writing-compiling-reviewing" and lack the intuitiveness of manipulation. Neither of these two approaches can well meet the needs for efficiency, intuitiveness, and iterative modification in scientific creation. To bridge this gap, we introduce VisPainter, a multi-agent framework for scientific illustration built upon the model context protocol. VisPainter orchestrates three specialized modules-a Manager, a Designer, and a Toolbox-to collaboratively produce diagrams compatible with standard vector graphics software. This modular, role-based design allows each element to be explicitly represented and manipulated, enabling true element-level control and any element can be added and modified later. To systematically evaluate the quality of scientific illustrations, we introduce VisBench, a benchmark with seven-dimensional evaluation metrics. It assesses high-information-density scientific illustrations from four aspects: content, layout, visual perception, and interaction cost. To this end, we conducted extensive ablation experiments to verify the rationality of our architecture and the reliability of our evaluation methods. Finally, we evaluated various vision-language models, presenting fair and credible model rankings along with detailed comparisons of their respective capabilities. Additionally, we isolated and quantified the impacts of role division, step control,and description on the quality of illustrations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。