评测图像编辑模型在复杂图表数据修改时的几何同步能力。
ChartSync: A Benchmark for Visuo-Logical Cascading Chart Editing

- 构建视觉逻辑级联编辑任务,测试文本与图形的联动更新。
- 870组图表数据中235个涉及几何耦合,揭示模型普遍失准。
- 专用于评估多模态模型对图表语义与结构的综合理解能力。
生成式图像编辑模型在处理需几何同步的数据图表时表现不佳。本文将此任务形式化为视觉逻辑级联编辑(VLCE),并提出 ChartSync 基准,通过程序化渲染管道构建专家验证的确定性图文耦合数据集。该数据集包含9类图表、4种任务类型,共870个三元组,其中235个为几何耦合的级联编辑实例。采用双层评估框架——结合客观视觉指标与视觉语言模型判别,全面评估低层保真度与多模态理解推理能力。对14个图像编辑模型和1个代码中介流程的评估显示,多数开源模型几何同步性能显著下降,仅两个前沿闭源模型展现出初步的VLCE能力,其残余错误主要源于语义孤立与背景污染。详细错误分析揭示了未来多模态架构所需的核心元能力。数据集与代码已公开于 https://github.com/kaka-yjk/ChartSyncCodebase。
原文摘要 · Abstract (English)
Generative image editing models struggle with structured statistical charts when data modifications require geometric synchronization. We formalize this task as Visuo-Logical Cascading Editing (VLCE). However, existing methods remain confined to localized text substitutions and struggle with dependency-aware cascading updates. To systematically evaluate this capability, we introduce ChartSync, an expert-validated benchmark constructed via a programmatic rendering pipeline that guarantees deterministic visuo-logical coupling for the ground truth. ChartSync comprises 870 triplets across 9 chart categories and 4 task types, including 235 geometry-coupled VLCE instances that specifically test cascading text-to-geometry synchronization. We further evaluate these instances via a two-tier framework combining objective visual metrics with a vision-language model judge paradigm to assess low-level fidelity alongside multimodal comprehension and reasoning. Evaluating 14 image editing models and one code-mediated pipeline reveals a nuanced capability gap: most open-source models suffer severe drops in geometric synchronization, while only two frontier proprietary models show emerging VLCE capability, with their residual errors mainly involving semantic isolation and background corruption. Our detailed error analysis deconstructs these failure paradigms to identify core meta-abilities for guiding future multimodal architectures. The ChartSync dataset and code are publicly released at https://github.com/kaka-yjk/ChartSyncCodebase.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。