用大模型生成文档描述，零样本实现高效图文检索。
SERVAL: Surprisingly Effective Zero-Shot Visual Document Retrieval Powered by Large Vision and Language Models
- 先用视觉语言模型生成图像文本描述，再用标准文本编码器嵌入。
- 在ViDoRe-v2上nDCG@5达63.4%，超越专用多向量编码器。
- 无需对比训练，适合大规模和多语言场景的快速部署。
视觉文档检索（VDR）通常采用专用双编码器进行文本到图像的检索，直接嵌入文档图像。本文重新审视一种零样本“生成-编码”流程：先由视觉语言模型对每张文档图像生成详细文本描述，再由标准文本编码器进行嵌入。在ViDoRe-v2基准上，该方法达到63.4%的nDCG@5，超越最强的专用多向量视觉文档编码器。该方法在大规模数据集上扩展性更好，且具备更广的多语言覆盖能力。分析表明，现代视觉语言模型能以足够细粒度捕捉复杂的文本与视觉线索，可作为可复用的语义代理。通过将模态对齐任务交由预训练视觉语言模型完成，本方法避免了计算密集的文本-图像对比训练，为未来VDR系统建立了强有力的零样本基线。
原文摘要 · Abstract (English)
Visual Document Retrieval (VDR) typically operates as text-to-image retrieval using specialized bi-encoders trained to directly embed document images. We revisit a zero-shot generate-and-encode pipeline: a vision-language model first produces a detailed textual description of each document image, which is then embedded by a standard text encoder. On the ViDoRe-v2 benchmark, the method reaches 63.4% nDCG@5, surpassing the strongest specialised multi-vector visual document encoder. It also scales better to large collections and offers broader multilingual coverage. Analysis shows that modern vision-language models capture complex textual and visual cues with sufficient granularity to act as a reusable semantic proxy. By offloading modality alignment to pretrained vision-language models, our approach removes the need for computationally intensive text-image contrastive training and establishes a strong zero-shot baseline for future VDR systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。