测试视觉语言模型读图中文字的能力，发现长文本阅读时表现大幅下降。
ReadBench: Measuring the Dense Text Visual Reading Ability of Vision-Language Models
- 将纯文本题库转为图文形式，保持题目不变测试模型理解力。
- 短文本阅读性能略有下降，长文本多页内容时表现急剧下滑。
- 适合研究多模态阅读、文档理解的学者和开发者使用。
近年来，大型视觉语言模型（VLMs）在联合处理图像与文本方面取得显著进展。然而，尽管已有大量基准评估视觉理解能力（如图表、配色、OCR任务等），对VLMs在图文混合场景下读取和推理密集文本内容的能力评估仍显不足。为此，我们提出ReadBench，一个专门用于评估VLMs读写理解能力的多模态基准。ReadBench将经典纯文本基准中的上下文转换为包含文本的图像，同时保留原始文本提示与问题。在主流VLMs上进行测试发现，对于简短图文输入，性能仅轻微下降；而面对较长、跨页的复杂文本情境，性能则显著衰退。实验还表明，文本分辨率对多模态表现影响微乎其微。这些结果揭示了当前VLMs在处理视觉呈现的长篇文字内容方面存在明显短板，这一能力对实际应用至关重要。ReadBench已开源：https://github.com/answerdotai/ReadBench。
原文摘要 · Abstract (English)
Recent advancements in Large Vision-Language Models (VLMs), have greatly enhanced their capability to jointly process text and images. However, despite extensive benchmarks evaluating visual comprehension (e.g., diagrams, color schemes, OCR tasks...), there is limited assessment of VLMs' ability to read and reason about text-rich images effectively. To fill this gap, we introduce ReadBench, a multimodal benchmark specifically designed to evaluate the reading comprehension capabilities of VLMs. ReadBench transposes contexts from established text-only benchmarks into images of text while keeping textual prompts and questions intact. Evaluating leading VLMs with ReadBench, we find minimal-but-present performance degradation on short, text-image inputs, while performance sharply declines for longer, multi-page contexts. Our experiments further reveal that text resolution has negligible effects on multimodal performance. These findings highlight needed improvements in VLMs, particularly their reasoning over visually presented extensive textual content, a capability critical for practical applications. ReadBench is available at https://github.com/answerdotai/ReadBench .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。