首个越南语信息图VQA基准,评估跨图像推理能力。
ViInfographicVQA: A Benchmark for Single and Multi-image Visual Question Answering on Vietnamese Infographics
- 构建多模态模型需融合文字识别与版面理解。
- 涵盖6747张真实信息图,20409个问答对,分单图与多图任务。
- 揭示当前模型在跨图整合与非跨度推理上的明显短板。
信息图视觉问答(InfographicVQA)评估模型读取和推理数据密集、版面复杂的视觉内容的能力,这类内容结合了文本、图表、图标和设计元素。相比场景文本或自然图像的VQA,信息图要求更强的OCR、版面理解以及数值与语义推理能力。我们提出了首个越南语信息图VQA基准——ViInfographicVQA,包含超过6747张真实世界的信息图和20409个由人工验证的问答对,覆盖经济、医疗、教育等多个领域。该基准包含两种评估设置:单图任务遵循传统VQA范式,每题基于一张信息图回答;多图任务需在多个语义相关的信息图间综合证据,据我们所知,这是首个针对越南语的跨图推理VQA评估。我们在该基准上测试了一系列近期视觉语言模型,发现性能差异显著,最严重的错误出现在涉及跨图整合与非跨度推理的多图问题上。ViInfographicVQA为越南语信息图VQA提供了基准结果,揭示了当前多模态模型在低资源语境下的局限性,推动未来布局感知与跨图推理方法的研究。
原文摘要 · Abstract (English)
Infographic Visual Question Answering (InfographicVQA) evaluates a model's ability to read and reason over data-rich, layout-heavy visuals that combine text, charts, icons, and design elements. Compared with scene-text or natural-image VQA, infographics require stronger integration of OCR, layout understanding, and numerical and semantic reasoning. We introduce ViInfographicVQA, the first benchmark for Vietnamese InfographicVQA, comprising over 6747 real-world infographics and 20409 human-verified question-answer pairs across economics, healthcare, education, and more. The benchmark includes two evaluation settings. The Single-image task follows the traditional setup in which each question is answered using a single infographic. The Multi-image task requires synthesizing evidence across multiple semantically related infographics and is, to our knowledge, the first Vietnamese evaluation of cross-image reasoning in VQA. We evaluate a range of recent vision-language models on this benchmark, revealing substantial performance disparities, with the most significant errors occurring on Multi-image questions that involve cross-image integration and non-span reasoning. ViInfographicVQA contributes benchmark results for Vietnamese InfographicVQA and sheds light on the limitations of current multimodal models in low-resource contexts, encouraging future exploration of layout-aware and cross-image reasoning methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。