通过三视角训练与问询驱动进化,提升图表转代码的准确率
CharTide: Data-Centric Chart-to-Code Generation via Tri-Perspective Tuning and Inquiry-Driven Evolution

- 分三路训练:视觉感知、代码逻辑、模态融合,解耦多模态学习
- 200万样本数据集,7B模型在多个数据集上超越专用基线
- 用原子问答验证生成图表一致性,避免依赖人工评分
图表转代码要求视觉精度和语法正确性。现有方法受限于数据质量:尽管有越来越多的图表-代码数据集,但简单堆叠同质数据对会使视觉感知与程序逻辑混淆,难以发挥多模态监督潜力。我们提出CharTide,一种全新的数据驱动框架,重构训练与对齐数据。首先,通过三视角调优构建200万样本数据集,将训练显式分为视觉感知、纯文本代码逻辑和模态融合三路,使7B模型仅用监督数据即超越专用基线。其次,将对齐重定义为数据验证问题而非启发式打分任务。为此,提出基于信息不变性的问询驱动强化学习框架:下游模型对原始图与生成图应给出一致回答。我们采用冻结的Inspector通过原子问答任务客观验证生成结果,以答案准确率提供可验证奖励信号。在ChartMimic、Plot2Code和ChartX上的实验表明,CharTide-7B/8B显著优于开源基线,超越GPT-4o,且媲美GPT-5。
原文摘要 · Abstract (English)
Chart-to-code generation demands strict visual precision and syntactic correctness from Vision-Language Models (VLMs). However, existing approaches are fundamentally constrained by data-centric limitations: despite the availability of growing chart-to-code datasets, simply scaling homogeneous chart-code pairs conflates visual perception with program logic, preventing models from fully leveraging the richness of multimodal supervision. We present CharTide, a novel data-centric framework that systematically redesigns both training and alignment data for chart-to-code generation. First, we construct a 2M-sample dataset via a Tri-Perspective Tuning strategy, explicitly decoupling training into visual perception, pure-text code logic, and modality fusion streams, enabling a 7B model to surpass specialized baselines using only supervised data. Second, we reformulate alignment as a data verification problem rather than a heuristic scoring task. To this end, we introduce an Inquiry-Driven RL framework grounded in the principle of information invariance: a downstream model should yield consistent answers to identical visual queries across both original and generated charts. Moving beyond rigid rule matching or VLM scoring, we employ a frozen Inspector to objectively verify generated charts through atomic QA tasks, providing verifiable reward signals based on answer accuracy. Experiments on ChartMimic, Plot2Code, and ChartX show that CharTide-7B/8B significantly outperforms open-source baselines, surpasses GPT-4o, and is competitive with GPT-5.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。