构建多语言长文档视觉检索基准，评估模型跨语言图文理解能力。
VisR-Bench: An Empirical Study on Visual Retrieval-Augmented Generation for Multilingual Long Document Understanding
- 设计涵盖16种语言的多模态检索数据集，支持图表、文本、表格三类问题。
- 超过3.5万组高质量问答对，覆盖1200篇长文档，支持细粒度评估。
- 揭示大模型在结构化表格和低资源语言上仍存短板，适合多语言研究者参考。
全球绝大多数组织数据以文档形式存储，视觉检索在挖掘文档集体智能中至关重要。然而，现有基准多局限于英文文档检索，或仅针对单页图像的多语言问答。为此，我们提出VisR-Bench，一个面向长文档的多语言、问题驱动的多模态检索基准。该基准包含超过35,000组高质量问答对，覆盖1,200篇文档，涵盖16种语言及三类问题（图表、文本、表格），实现多样化的语言与问题覆盖。不同于以往数据集，我们引入无明确答案的查询，避免模型依赖表面关键词匹配。我们评估了文本基方法、多模态编码器及多模态大模型（MLLMs）的表现，结果表明：尽管MLLMs显著优于传统方法，但在结构化表格和低资源语言上仍表现不佳，揭示了多语言视觉检索的关键挑战。
原文摘要 · Abstract (English)
Most organizational data in this world are stored as documents, and visual retrieval plays a crucial role in unlocking the collective intelligence from all these documents. However, existing benchmarks focus on English-only document retrieval or only consider multilingual question-answering on a single-page image. To bridge this gap, we introduce VisR-Bench, a multilingual benchmark designed for question-driven multimodal retrieval in long documents. Our benchmark comprises over 35K high-quality QA pairs across 1.2K documents, enabling fine-grained evaluation of multimodal retrieval. VisR-Bench spans sixteen languages with three question types (figures, text, and tables), offering diverse linguistic and question coverage. Unlike prior datasets, we include queries without explicit answers, preventing models from relying on superficial keyword matching. We evaluate various retrieval models, including text-based methods, multimodal encoders, and MLLMs, providing insights into their strengths and limitations. Our results show that while MLLMs significantly outperform text-based and multimodal encoder models, they still struggle with structured tables and low-resource languages, highlighting key challenges in multilingual visual retrieval.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。