对比视觉与OCR两种文档检索方式，发现后者更适应不同质量文档。
Lost in OCR Translation? Vision-Based Approaches to Robust Document Retrieval
- 用视觉嵌入直接处理文档图像，跳过易出错的OCR步骤
- OCR方案在未见文档上表现更优，尤其在低质量扫描件中
- 适合需要跨类型文档泛化的实际应用部署
检索增强生成（RAG）通过引入外部文档提升大语言模型（LLM）的可靠性。传统RAG依赖OCR将扫描文档转为文本，但即使最先进的OCR在复杂或劣质文档上仍会出错。最近的视觉-语言方法如ColPali，可直接对文档图像进行视觉嵌入，无需OCR。本研究系统比较了基于视觉的RAG系统（ColPali）与传统的OCR管道（使用Llama 3.2 90B和Nougat OCR），在不同文档质量下的表现。除常规检索准确率外，还引入语义回答评估基准，衡量端到端问答性能。结果表明：虽然视觉RAG在训练过的文档上表现良好，但基于OCR的RAG更能泛化到未见且质量各异的文档。研究揭示了计算效率与语义准确性之间的关键权衡，为生产环境中选择依赖OCR或纯视觉的文档检索系统提供了实用建议。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) has become a popular technique for enhancing the reliability and utility of Large Language Models (LLMs) by grounding responses in external documents. Traditional RAG systems rely on Optical Character Recognition (OCR) to first process scanned documents into text. However, even state-of-the-art OCRs can introduce errors, especially in degraded or complex documents. Recent vision-language approaches, such as ColPali, propose direct visual embedding of documents, eliminating the need for OCR. This study presents a systematic comparison between a vision-based RAG system (ColPali) and more traditional OCR-based pipelines utilizing Llama 3.2 (90B) and Nougat OCR across varying document qualities. Beyond conventional retrieval accuracy metrics, we introduce a semantic answer evaluation benchmark to assess end-to-end question-answering performance. Our findings indicate that while vision-based RAG performs well on documents it has been fine-tuned on, OCR-based RAG is better able to generalize to unseen documents of varying quality. We highlight the key trade-offs between computational efficiency and semantic accuracy, offering practical guidance for RAG practitioners in selecting between OCR-dependent and vision-based document retrieval systems in production environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。