测试视觉语言模型识别误导性图表的能力，发现其对设计缺陷敏感但难辨逻辑谬误。
When Visuals Aren't the Problem: Evaluating Vision-Language Models on Misleading Data Visualizations
- 构建含人为误导标题的图表数据集，按推理与设计错误分类评估
- 模型对轴截断等设计问题检测准确率高，对因果误读等逻辑错误识别差
- 揭示当前VLM在辨别复杂误导信息上的局限，适合研究可信AI的学者参考
可视化有助于传达数据洞察，但欺骗性数据呈现可能扭曲解读并传播错误信息。尽管近期视觉语言模型（VLMs）在多种图表理解任务中表现良好，其对误导性可视化——尤其是由标题中的细微推理错误引发的误导——的识别能力仍不明确。本文基于细粒度的推理错误分类（如樱桃选样、因果推断）和可视化设计错误（如轴截断、双轴、不当编码），评估VLMs的表现。我们构建了一个基准数据集，结合真实世界可视化与人工撰写的误导性标题，旨在系统分析不同错误类型及误导模态下的模型表现。在多个商用和开源VLM上测试发现，模型对可视化设计错误的检测远优于推理类误导，且常将非误导性图表误判为欺骗性。本工作填补了粗粒度误导检测与具体错误归因之间的空白。
原文摘要 · Abstract (English)
Visualizations help communicate data insights, but deceptive data representations can distort their interpretation and propagate misinformation. While recent Vision Language Models (VLMs) perform well on many chart understanding tasks, their ability to detect misleading visualizations, especially when deception arises from subtle reasoning errors in captions, remains poorly understood. Here, we evaluate VLMs on misleading visualization-caption pairs grounded in a fine-grained taxonomy of reasoning errors (e.g., Cherry-picking, Causal inference) and visualization design errors (e.g., Truncated axis, Dual axis, inappropriate encodings). To this end, we develop a benchmark that combines real-world visualization with human-authored, curated misleading captions designed to elicit specific reasoning and visualization error types, enabling controlled analysis across error categories and modalities of misleadingness. Evaluating many commercial and open-source VLMs, we find that models detect visual design errors substantially more reliably than reasoning-based misinformation, and frequently misclassify non-misleading visualizations as deceptive. Overall, our work fills a gap between coarse detection of misleading content and the attribution of the specific reasoning or visualization errors that give rise to it.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。