构建多图表问答基准，测试模型跨图表推理能力。
MultiChartQA: Benchmarking Vision-Language Models on Multi-Chart Problems
- 设计四类多图表任务，涵盖并列、比较、顺序推理等复杂场景。
- 多模型测试显示人类表现远超现有MLLMs，差距显著。
- 适合研究视觉语言模型在真实数据场景中的推理能力。
多模态大语言模型（MLLMs）在视觉问答和图表理解等任务中展现出强大能力，但现有图表类基准难以反映真实世界中多图表场景的复杂性。当前基准多聚焦单图表任务，忽视了从多个图表中提取并整合信息所需的多跳推理，而这在实际应用中至关重要。为此，我们提出MultiChartQA，一个评估MLLMs在四大关键领域能力的基准：直接问答、并列问答、比较推理和顺序推理。对多种MLLM的评估显示，其表现与人类存在显著差距。这些结果揭示了多图表理解的挑战，并展示了MultiChartQA推动该领域发展的潜力。代码与数据已公开于 https://github.com/Zivenzhu/Multi-chart-QA。
原文摘要 · Abstract (English)
Multimodal Large Language Models (MLLMs) have demonstrated impressive abilities across various tasks, including visual question answering and chart comprehension, yet existing benchmarks for chart-related tasks fall short in capturing the complexity of real-world multi-chart scenarios. Current benchmarks primarily focus on single-chart tasks, neglecting the multi-hop reasoning required to extract and integrate information from multiple charts, which is essential in practical applications. To fill this gap, we introduce MultiChartQA, a benchmark that evaluates MLLMs' capabilities in four key areas: direct question answering, parallel question answering, comparative reasoning, and sequential reasoning. Our evaluation of a wide range of MLLMs reveals significant performance gaps compared to humans. These results highlight the challenges in multi-chart comprehension and the potential of MultiChartQA to drive advancements in this field. Our code and data are available at https://github.com/Zivenzhu/Multi-chart-QA
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。