arXiv:2607.04726cs.CLcs.LG2026-07

让图表生成模型只依赖可观察数据,提升代码准确性。

What You See Is What You Get: Observation-Aligned Supervision for Chart-to-Code Generation

论文配图:What You See Is What You Get: Observation-Aligned Supervision for Chart-to-Code Generation
图 1 · 摘自论文原文
  • 用可观察量替代隐藏变量作为训练目标
  • 在多个图表类型上显著提升可还原性
  • 适合做图表转代码的开发者与研究者

图表转代码通常采用监督微调,隐含假设真实代码是完全可观测的目标。我们指出这一假设常不成立:许多图表程序包含无法从渲染图像中唯一还原的隐藏原始变量。这种系统性隐变量-观测不匹配表现为三种形式:聚合导致的失真(原始数据被简化为统计量或分组质量)、归一化导致的失真(绝对尺度丢失)、投影导致的失真(高维信息在视觉投影中损失)。这些不匹配引入目标模糊性,迫使模型补全图像未提供的信息。我们提出观察对齐监督,将隐变量替换为可视约束量:针对箱线图使用箱统计量,直方图使用分组权重,饼图使用楔形比例,并通过受控3D散点实验验证投影失真。在ChartMimic和ChartX数据集上,多款VLM模型经重写监督后,在可观测值恢复任务中表现提升,包括在不可执行评估下的表现。

原文摘要 · Abstract (English)

Chart-to-code generation is commonly trained with supervised fine-tuning on reference plotting scripts, implicitly treating the gold code as a fully observable target. We argue that this assumption is often invalid: many chart programs contain latent raw variables that cannot be uniquely recovered from the rendered image. We identify this systematic latent--observation mismatch with three forms: aggregation-induced mismatch, where raw samples are reduced to summary statistics or bin-level mass; normalization-induced mismatch, where absolute scale is removed; and projection-induced mismatch, where higher-dimensional information is lost through visual projection. These mismatches introduce target ambiguity and require models to complete information unsupported by the image. We propose Observation-Aligned Supervision, which replaces latent targets with visually constrained quantities. We instantiate it for boxplots, histograms, and pie charts using box statistics, bin weights, and wedge proportions, respectively, and study projection-induced mismatch through a controlled 3D scatter experiment. Across multiple VLMs on ChartMimic and ChartX, our rewritten supervision improves observable-value recovery, including under both-executable evaluation.

图表生成代码生成视觉对齐监督学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。