用代码生成图表，构建大规模训练数据集，推动图表理解模型发展。
ChartGen: Scaling Chart Understanding Via Code-Guided Synthetic Chart Generation
- 通过视觉语言模型重建图表为代码，并用大模型迭代优化脚本。
- 生成22.5万对图表-代码数据，覆盖27种图表类型和11个绘图库。
- 开源完整流程与数据集，适合研究图表理解与视觉编程的团队使用。
图表到代码的重建任务——从图表图像中恢复可执行的绘图脚本——能有效评估模型将数据可视化转化为精确、机器可读形式的能力。然而，现有多数多模态基准主要聚焦于回答图表问题或进行摘要。为此，我们提出ChartGen，一种全自动的代码引导合成图表生成管道。该方法从种子图表图像出发，(i) 利用视觉语言模型（VLM）将每张图像重构为Python脚本，(ii) 再通过面向代码的大语言模型（LLM）迭代增强脚本。基于此，我们从1.3万张种子图像生成了22.5万组唯一的图表-代码对，并构建了一个开源合成数据集，涵盖27种图表类型、11个绘图库及多种数据模态（图像、代码、文本、CSV、DocTags）。从中筛选出4.3千对用于评估的独立测试集，测试六款开源大模型（参数量3B - 26B），揭示当前模型仍有巨大提升空间。我们公开发布该生成管道、提示模板及完整数据集，以加速鲁棒图表理解与视觉条件下的代码生成研究：https://github.com/SD122025/ChartGen/
原文摘要 · Abstract (English)
Chart-to-code reconstruction -- the task of recovering executable plotting scripts from chart images -- provides important insights into a model's ability to ground data visualizations in precise, machine-readable form. Yet many existing multimodal benchmarks largely focus primarily on answering questions about charts or summarizing them. To bridge this gap, we present ChartGen, a fully-automated pipeline for code-guided synthetic chart generation. Starting from seed chart images, ChartGen (i) prompts a vision-language model (VLM) to reconstruct each image into a python script, and (ii) iteratively augments that script with a code-oriented large language model (LLM). Using ChartGen, we create 222.5K unique chart-image code pairs from 13K seed chart images, and present an open-source synthetic chart dataset covering 27 chart types, 11 plotting libraries, and multiple data modalities (image, code, text, CSV, DocTags). From this corpus, we curate a held-out chart-to-code evaluation subset of 4.3K chart image-code pairs, and evaluate six open-weight VLMs (3B - 26B parameters), highlighting substantial room for progress. We release the pipeline, prompts, and the dataset to help accelerate efforts towards robust chart understanding and vision-conditioned code generation: https://github.com/SD122025/ChartGen/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。