评测大模型生成图表的可信度，发现其常隐藏数据幻觉。
DEEPCHART: How Far are LLMs from Faithful Data-Science Chart Generation?
- 构建三阶段流程：提取数据、推理计算、生成图表
- 1482个真实场景实例中，多数图表存在数据错误
- 适合关注可信AI可视化与模型评估的研究者
真实数据科学工作流中的可信图表生成需基于分散证据进行数据提取、计算图表所需数值，并准确渲染。当前大模型虽能生成视觉上合理的指令遵循图表，但在长篇、嘈杂和多模态情境下，数据级幻觉难以检测。为此，我们提出DEEPCHART，一个包含1,482个任务条件化图表生成实例的专家标注基准，数据源自真实科学论文、金融报告和生态报告。DEEPCHART将图表生成建模为提取-推理-可视化三阶段流程，分阶段评估源数据提取、衍生数据推理和图表渲染能力。对前沿模型的实验表明，视觉上看似合理的图表常隐含数据幻觉，在真实复杂场景中，数据提取与推理错误普遍存在。这表明仅扩大上下文窗口不足；实现可信图表生成还需可靠证据提取与量化推理能力。相关基准与资源已开源：https://github.com/tangdouer1005/DeepChart。
原文摘要 · Abstract (English)
Faithful chart generation in real-world data-science workflows requires grounding visualizations in scattered evidence, computing chart-ready quantities, and rendering them accurately. Modern LLMs can produce visually plausible, instruction-compliant charts, yet data-level hallucinations remain difficult to detect in long, noisy, and multimodal contexts. To measure this gap, we introduce DEEPCHART, an expert-annotated benchmark of 1,482 task-conditioned chart-generation instances drawn from real-world scientific papers, financial filings, and ecosystem reports. DEEPCHART formulates chart generation as an Extract--Reason--Visualize pipeline and evaluates source-data extraction, derived-data reasoning, and chart rendering stage by stage. Experiments with state-of-the-art models show that visually plausible charts often conceal data-level hallucinations, with extraction and reasoning errors common in realistic long and multimodal settings. These findings suggest that larger context windows alone are insufficient; faithful chart generation also requires reliable evidence extraction and quantitative reasoning before rendering. Our benchmark and associated resources are available at https://github.com/tangdouer1005/DeepChart.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。