让大模型解释图表分析时依赖了哪部分数据,提升可信度。
RADAR: A Reasoning-Guided Attribution Framework for Explainable Visual Data Analysis
- 通过推理引导定位模型决策依据的图表区域。
- 相比基线方法,归因准确率提升15%,答案与真实答案匹配度达0.90。
- 适合关注可解释性、需要验证模型判断的研究者和工程师。
图表是跨领域定量分析与决策的核心工具,要求精准解读与数学推理。多模态大语言模型(MLLMs)虽能自动处理图表、回答问题并生成摘要,但缺乏对决策依据的可视化,存在黑箱问题,阻碍实际信任与应用。本文提出RADAR,一种半自动框架,构建包含17,819个样本的基准数据集,涵盖图表、问题、推理步骤及归因标注。我们还提出一种针对图表数学推理的归因方法。实验表明,该推理引导方法使归因准确率提升15%,答案生成的BERTScore达~0.90,表明与真实答案高度一致。此进展推动可解释、可信的图表分析系统发展,使用户可通过推理与归因验证模型决策。
原文摘要 · Abstract (English)
Data visualizations like charts are fundamental tools for quantitative analysis and decision-making across fields, requiring accurate interpretation and mathematical reasoning. The emergence of Multimodal Large Language Models (MLLMs) offers promising capabilities for automated visual data analysis, such as processing charts, answering questions, and generating summaries. However, they provide no visibility into which parts of the visual data informed their conclusions; this black-box nature poses significant challenges to real-world trust and adoption. In this paper, we take the first major step towards evaluating and enhancing the capabilities of MLLMs to attribute their reasoning process by highlighting the specific regions in charts and graphs that justify model answers. To this end, we contribute RADAR, a semi-automatic approach to obtain a benchmark dataset comprising 17,819 diverse samples with charts, questions, reasoning steps, and attribution annotations. We also introduce a method that provides attribution for chart-based mathematical reasoning. Experimental results demonstrate that our reasoning-guided approach improves attribution accuracy by 15% compared to baseline methods, and enhanced attribution capabilities translate to stronger answer generation, achieving an average BERTScore of $\sim$ 0.90, indicating high alignment with ground truth responses. This advancement represents a significant step toward more interpretable and trustworthy chart analysis systems, enabling users to verify and understand model decisions through reasoning and attribution.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。