构建分层图表到代码基准,评估大模型理解与生成图表能力。
From Charts to Code: A Hierarchical Benchmark for Multimodal Models
- 从用户视角设计三阶任务:复现、编辑、长表转图。
- 25个顶尖模型在编辑任务中代码正确率仅0.57,图表质量仅0.22。
- 首个兼顾真实场景与难度递进的多模态图表生成评测体系。
我们提出Chart2Code,一个面向大型多模态模型(LMMs)的图表理解与代码生成评测基准。该基准从用户实际需求出发,涵盖多样化现实场景并逐步提升任务难度,分为三个层级:一级(图表复现)根据参考图和用户查询重建图表;二级(图表编辑)涉及类型变更或元素增删等复杂操作;三级(长表转图)要求将信息密集的长表格转化为符合指令的准确图表。目前共包含2,023个任务,覆盖22种图表类型,并配备多层次评估指标,同时衡量代码正确性与渲染图表的视觉保真度。我们对25个前沿LMMs进行了测试,包括GPT-5、Qwen2.5-VL、InternVL3/3.5、MiMo-VL和Seed-1.6-VL等开源与闭源模型。实验结果表明,即使是最先进的模型GPT-5在编辑任务中的代码评估平均分仅为0.57,图表质量评估平均分仅0.22,凸显了该基准的挑战性。我们期望此基准能推动多模态推理进步,促进更鲁棒、通用的LMMs发展。代码与数据已公开于Chart2Code。
原文摘要 · Abstract (English)
We introduce Chart2Code, a new benchmark for evaluating the chart understanding and code generation capabilities of large multimodal models (LMMs). Chart2Code is explicitly designed from a user-driven perspective, capturing diverse real-world scenarios and progressively increasing task difficulty. It consists of three levels: Level 1 (Chart Reproduction) reproduces charts from a reference figure and user query; Level 2 (Chart Editing) involves complex modifications such as changing chart types or adding elements; and Level 3 (Long-Table to Chart Generation) requires models to transform long, information-dense tables into faithful charts following user instructions. To our knowledge, this is the first hierarchical benchmark that reflects practical chart2code usage while systematically scaling task complexity. In total, Chart2Code contains 2,023 tasks across 22 chart types, paired with multi-level evaluation metrics that assess both code correctness and the visual fidelity of rendered charts. We benchmark 25 state-of-the-art (SoTA) LMMs, including both proprietary and the latest open-source models such as GPT-5, Qwen2.5-VL, InternVL3/3.5, MiMo-VL, and Seed-1.6-VL. Experimental results demonstrate that even the SoTA model GPT-5 averages only 0.57 on code-based evaluation and 0.22 on chart-quality assessment across the editing tasks, underscoring the difficulty of Chart2Code. We anticipate this benchmark will drive advances in multimodal reasoning and foster the development of more robust and general-purpose LMMs. Our code and data are available on Chart2Code.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。