探究大模型生成图表解读的可信度，发现上下文影响判断类型与准确性
Chart-Supported or Model-Supplied? Examining MLLM-Generated Claims for Accessible Visualization
- 对比不同输入条件，分析模型生成的直接/推断/推测类判断
- 有图表上下文时模型更倾向给出直接结论，且数值一致性提升
- 提示词设计难保证谨慎表达，多数结论仍属推测
多模态大语言模型（MLLMs）可将可视化模式与外部因果、后果及领域知识关联，但其解释的证据基础常不明确。我们对来自四个来源的102个可视化、三种MLLM及四种输入条件（图像可访问性、特定图表上下文可用性、隐去上下文的提示）进行了探索性研究。在1,224条描述中，分析了模型标注的直接（DIRECT）、推断（DERIVED）和推测（SPECULATIVE）标签，并开展自动化数值一致性审计。结果显示，可访问的图表上下文使Gemini和GPT更倾向于生成直接判断，部分模型的数值一致性得到改善；在完整上下文中加入图像并未带来一致的数值优势，而隐去上下文提示也未可靠促使模型使用更谨慎的语言。提示词定义的‘现实意义’部分仍以推测为主。这些结果推动构建能区分证据支持与模型推断的可访问描述系统。
原文摘要 · Abstract (English)
Multimodal large language models (MLLMs) can connect visualization patterns to external causes, consequences, and domain knowledge, but the evidential basis of these interpretations is often unclear. We present an exploratory study of 102 visualizations from four sources, three MLLMs, and four input conditions that vary access to the image, source-specific accessible chart context, and withheld-context framing. Across 1,224 descriptions, we analyze model-attributed DIRECT, DERIVED, and SPECULATIVE labels and conduct an automated audit of numeric agreement. Accessible chart context shifted Gemini and GPT toward DIRECT claims and improved numeric agreement for some models. Adding the image to the full context did not yield a consistent numeric benefit, and the withheld-context prompt did not reliably increase cautious language. The prompt-defined Real-World Significance section remained predominantly SPECULATIVE. These results motivate accessible description systems that distinguish claims supported by supplied evidence from model-supplied interpretation
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。