评测大模型在图表理解中的幻觉问题，发现顶尖模型准确率不足四成。
ChartHal: A Fine-grained Framework Evaluating Hallucination of Large Vision Language Models in Chart Understanding
- 构建细粒度幻觉分类框架，覆盖图表理解中常见错误类型。
- 在1062个样本上测试，顶级模型如GPT-5准确率仅34.46%。
- 揭示图表外信息或矛盾问题最易引发幻觉，适合评估与改进模型可靠性。
大型视觉语言模型（LVLMs）近期取得显著进展，但幻觉仍是关键障碍，尤其在需要复杂感知与认知能力的图表理解任务中，对事实准确性要求极高。现有研究多分别关注幻觉或图表理解，其交叉领域仍缺乏探索。为此，我们提出ChartHal，一个包含1,062个经人工验证样本的基准数据集，具有细粒度的幻觉场景分类体系。评估显示，当前最先进的LVLM在ChartHal上表现严重不足，包括GPT-5和o4-mini等专有模型，准确率分别仅为34.46%和22.79%。进一步分析表明，涉及图表中不存在或矛盾信息的问题极易诱发幻觉，凸显亟需更鲁棒的缓解策略。代码与数据已公开于https://github.com/ymcui/ChartHal。
原文摘要 · Abstract (English)
Large Vision-Language Models (LVLMs) have recently demonstrated remarkable progress, yet hallucination remains a critical barrier, particularly in chart understanding, which requires sophisticated perceptual and cognitive abilities as well as rigorous factual accuracy. While prior work has investigated hallucinations and chart comprehension independently, their intersection remains largely unexplored. To address this gap, we present ChartHal, a benchmark that features a fine-grained taxonomy of hallucination scenarios in chart understanding, along with a human-validated dataset of 1,062 samples. Our evaluation shows that state-of-the-art LVLMs suffer from severe hallucinations on ChartHal, including proprietary models such as GPT-5 and o4-mini, which achieve only 34.46% and 22.79% accuracy, respectively. Further analysis reveals that questions involving information absent from or contradictory to charts are especially likely to trigger hallucinations, underscoring the urgent need for more robust mitigation strategies. Code and data are available at https://github.com/ymcui/ChartHal .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。