arXiv:2605.27298cs.CL2026-05

通过多次采样融合提升图表数据提取准确率

Self-Ensembling Vision-Language Models for Chart Data Extraction

论文配图:Self-Ensembling Vision-Language Models for Chart Data Extraction
图 1 · 摘自论文原文
  • 对同一图表多次生成表格,按单元格取中位数融合结果
  • 在复杂图表上相对提升23%准确率,支持自动停止采样
  • 适合需要高可靠数据提取的科研与数据分析场景

图表能有效传递量化信息,但其数据常以图像形式存在,难以复用和分析。手动数字化耗时且易错,推动了自动图表转表格的需求。现有方法使用专用视觉语言模型(VLM),但在数据点多或风格差异大的图表上表现仍不足。本文提出一种VLM自集成方法:对同一图表图像多次采样生成多个表格,再在单元格层面进行对齐并取数值中位数,形成更准确的共识表。方法还包含收敛检测机制,当聚合结果稳定时停止采样,并基于样本间离散度估算不确定性,帮助用户评估提取可靠性。由于现有基准图表较简单,改进空间有限,我们引入新基准WB-ChartExtract,基于世界银行数据构建,其图表平均包含7倍于ChartQA的数据点。在ChartQA与WB-ChartExtract上,本方法均优于单次输出,尤其在后者上实现最高23%的相对准确率提升。该方法有助于释放图像中长期被封闭的表格数据,促进后续分析与复用。

原文摘要 · Abstract (English)

Charts effectively convey quantitative information, but the underlying data are often locked in image form, hindering reuse and analysis. Manually digitizing charts is time-consuming and error-prone, motivating automatic chart-to-table extraction. Recent approaches use specialized vision-language models (VLMs), yet performance still lags on charts with many datapoints or substantial stylistic variation. We propose a VLM self-ensembling method that repeatedly samples multiple tabular outputs from the same VLM for a fixed chart image and aggregates them at the level of individual table cells. We align candidate tables and take per-cell medians over numerical values to produce a more accurate consensus table. Our method also includes convergence detection to stop sampling once the aggregated table stabilizes, and uncertainty estimation based on dispersion across samples to help users assess extraction reliability. Because existing chart extraction benchmarks contain relatively simple plots with limited room for improvement, we introduce WB-ChartExtract, a new benchmark built from World Bank data with more complex and stylistically diverse charts; on average, its charts contain 7 times more datapoints than those in the ChartQA benchmark. Across both ChartQA and WB-ChartExtract, our approach improves extraction accuracy over single-pass VLM outputs, yielding up to 23% relative improvement on WB-ChartExtract after ensembling. More broadly, our method helps unlock tabular data previously siloed in chart images, enabling downstream analysis and reuse.

图表提取视觉语言模型数据挖掘

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。