用代码改图生成反事实数据，让模型更懂图表细微变化的含义。
Learning More from Less: Exploiting Counterfactuals for Data-Efficient Chart Understanding
- 通过修改生成图表的代码，自动构造反事实样本增强训练。
- 在5个基准上用更少数据达到顶尖模型水平。
- 适合做高效图表理解研究或小样本场景应用。
视觉语言模型（VLMs）在图表理解方面进展显著，主要依赖于大规模合成数据的监督微调（SFT）。然而，单纯扩大数据规模效率低下，且忽略了图表的核心特性：图表是程序生成的视觉产物，代码中微小的视觉改动可引发语义和正确答案的剧烈变化。学习这种反事实敏感性需要模型区分精细的视觉差异，但标准SFT独立处理每个样本，缺乏足够监督来培养此能力。为此，我们提出ChartCF，一种数据高效的训练框架，以增强反事实敏感性。ChartCF包含：(1) 通过代码修改生成反事实数据的流水线；(2) 基于图表相似性的数据选择策略，过滤过难样本以提升训练效率；(3) 跨文本与视觉模态的多模态偏好优化。在五个基准上的实验表明，ChartCF在使用显著更少训练数据的情况下，性能优于或媲美强效专用图表VLM。
原文摘要 · Abstract (English)
Vision-Language Models (VLMs) have demonstrated remarkable progress in chart understanding, largely driven by supervised fine-tuning (SFT) on increasingly large synthetic datasets. However, scaling SFT data alone is inefficient and overlooks a key property of charts: charts are programmatically generated visual artifacts, where small, code-controlled visual changes can induce drastic shifts in semantics and correct answers. Learning this counterfactual sensitivity requires VLMs to discriminate fine-grained visual differences, yet standard SFT treats training instances independently and provides limited supervision to enforce this behavior. To address this, we introduce ChartCF, a data-efficient training framework designed to enhance counterfactual sensitivity. ChartCF consists of: (1) a counterfactual data synthesis pipeline via code modification, (2) a chart similarity-based data selection strategy that filters overly difficult samples for improved training efficiency, and (3) multimodal preference optimization across both textual and visual modalities. Experiments on five benchmarks show that ChartCF achieves superior or comparable performance to strong chart-specific VLMs while using significantly less training data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。