视觉误导让大模型看错图表，揭示AI解读可视化时的脆弱性
The Perils of Chart Deception: How Misleading Visualizations Affect Vision-Language Models
- 测试16000+图表响应，评估10个大模型对8类误导图表的识别能力
- 多数模型被误导，相同数据下得出错误结论，误判率超70%
- 适合关注AI可信赖性、信息可视化安全的研究者与决策者
信息可视化是帮助用户快速发现模式、趋势和异常的重要工具，但若包含截断或反转坐标轴、不合理3D效果等欺骗性设计元素，可能误导观众并传播错误信息。尽管部分手法明显，但许多设计微妙地操控感知却仍保持表面合理性。随着视觉语言模型（VLMs）被广泛用于解读图表，尤其在非专业用户中，理解其对误导性设计的敏感性至关重要。本研究深入评估了10种不同VLMs在8类典型误导图表上的表现，分析超过16,000次响应结果。结果显示，大多数VLMs容易被误导，即使原始数据未变，也会产生错误解读。研究凸显了在VLM中建立抗视觉误导机制的紧迫性。
原文摘要 · Abstract (English)
Information visualizations are powerful tools that help users quickly identify patterns, trends, and outliers, facilitating informed decision-making. However, when visualizations incorporate deceptive design elements-such as truncated or inverted axes, unjustified 3D effects, or violations of best practices-they can mislead viewers and distort understanding, spreading misinformation. While some deceptive tactics are obvious, others subtly manipulate perception while maintaining a facade of legitimacy. As Vision-Language Models (VLMs) are increasingly used to interpret visualizations, especially by non-expert users, it is critical to understand how susceptible these models are to deceptive visual designs. In this study, we conduct an in-depth evaluation of VLMs' ability to interpret misleading visualizations. By analyzing over 16,000 responses from ten different models across eight distinct types of misleading chart designs, we demonstrate that most VLMs are deceived by them. This leads to altered interpretations of charts, despite the underlying data remaining the same. Our findings highlight the need for robust safeguards in VLMs against visual misinformation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。