构建首个基于维基百科的视觉问答基准，要求结合图文与外部知识答题。
WikiVQABench: A Knowledge-Grounded Visual Question Answering Benchmark from Wikipedia and Wikidata
- 用大模型生成候选题集，再经人工审核确保事实正确性
- 15个模型测试准确率从24.7%到75.6%，差距显著
- 适合评估视觉语言模型的知识推理能力
视觉问答基准大多聚焦仅靠图像内容即可解答的任务，但现实场景中常需图像外的外部知识。本文提出WikiVQABench，通过系统整合维基百科图片、文章标题及维基数据的结构化知识构建知识增强型视觉问答基准。采用大语言模型（LLMs）生成候选多选题-答案对，所有样本均由人工标注者审核，确保事实准确性、图文一致性，并保证每个问题需结合外部知识才能正确回答。该基准包含大量维基百科图像及其精心设计的多选题，用于评测知识感知型视觉语言模型（VLMs）。对15个VLM（参数量256M–90B）的评估显示，准确率范围为24.7%至75.6%，表明该基准能有效区分模型在知识密集型推理上的能力差异。数据集与评测代码已公开。
原文摘要 · Abstract (English)
Visual Question Answering (VQA) benchmarks have largely emphasized perception-based tasks that can be solved from visual content alone. In contrast, many real-world scenarios require external knowledge that is not directly observable in the image to answer correctly. We introduce WikiVQABench, a human-curated knowledge-grounded VQA benchmark constructed by systematically combining Wikipedia images, their associated article captions, and structured knowledge from Wikidata. Our pipeline uses large language models (LLMs) to generate candidate multiple-choice image-question-answer sets. All generated instances are subsequently reviewed and curated by human annotators to ensure factual correctness, visual-text consistency, and that each question requires external knowledge in addition to visual evidence for correct resolution. WikiVQABench comprises a substantial collection of Wikipedia images with curated multiple-choice questions designed to benchmark knowledge-aware vision-language models (VLMs). Evaluation of fifteen VLMs (256M-90B parameters) reveals a wide performance range (24.7%-75.6% accuracy), demonstrating that the benchmark effectively discriminates model capabilities on knowledge-intensive reasoning. The dataset and benchmarking code are publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。