arXiv:2608.17889cs.IRcs.AI2026-08

评测智能体在图文文档检索中的表现,发现视觉信息能显著提升效果。

VisDocAgentBench: Benchmarking Agents for Visually Rich Document Retrieval

论文配图:VisDocAgentBench: Benchmarking Agents for Visually Rich Document Retrieval
图 1 · 摘自论文原文
  • 构建闭源文档检索基准,对比静态与智能体检索策略
  • 视觉检索使召回率从37.5%提升至67.5%,尤其在复杂推理中优势明显
  • 适合研究多模态检索、智能体系统与文档理解的开发者和学者

图文丰富文档的语义相关性依赖语言、版式、结构化视觉元素及语料上下文,但现有检索评估多基于单次查询-页面匹配。主流代理搜索评测侧重下游问答或报告生成,忽视了在迭代证据获取下的文档排序。我们提出VisDocAgentBench,一个闭源基准,用于比较静态与代理式检索在统一排序输出协议下的表现。该数据集包含100份文档的2,375页,以及120个目标查询,覆盖直接、单桥接和双桥接证据结构。通过保持关系的构建方式生成语义、关系与视觉查询,经全文档审查与难负样本验证。强晚交互视觉检索器在直接项上达到97.50% Recall@1,但在双桥接项上仅2.50%,暴露了仅依赖查询-目标匹配的局限。代理可部分弥补损失,但规划器选择与检索表示仍起决定作用。所有规划器在使用视觉检索时表现更优,其最佳R@1达67.50%,而仅用OCR文本为37.50%。消融实验表明迭代搜索与页面检查是关键能力,提供完整支持上下文可改善两类路径的排名。轨迹分析定位剩余损失源于目标发现、候选评估与证据角色整合。这些发现推动兼具模态保真发现与证据导向验证的检索代理发展。

原文摘要 · Abstract (English)

Visually rich documents encode relevance through language, layout, structured visual elements, and corpus context, yet retrieval is typically evaluated by one-shot query--page matching. Agentic-search benchmarks usually score downstream question answering or report generation, leaving document ranking under iterative evidence acquisition underexplored. We introduce VisDocAgentBench, a closed-corpus benchmark comparing static and agentic retrieval under a shared ranked-output contract. It contains 2,375 pages from 100 documents and 120 unique-target queries balanced across direct, one-bridge, and two-bridge evidence structures. Relation-preserving construction yields semantic, relational, and visual queries, followed by full-document review and hard-negative validation. A strong late-interaction visual retriever reaches 97.50% Recall@1 on direct items but 2.50% on two-bridge items, exposing the limits of query--target matching when relevance depends on corpus context. Agents recover much of this loss, but planner choice and retrieval representation remain decisive. Every planner performs better with visual retrieval, whose best R@1 reaches 67.50% versus 37.50% for OCR-text. Ablations identify iterative search and page inspection as consequential capabilities, and providing the complete support context improves ranking on both routes. Trace analysis localizes the remaining losses to target discovery, candidate examination, and evidence-role integration. These findings motivate retrieval agents that combine modality-preserving discovery with evidence-directed verification.

文档检索多模态智能体评测基准

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。