用多模态大模型提升OCR文本质量，优化中文非叙事文档检索效果
KAP: MLLM-assisted OCR Text Enhancement for Hybrid Retrieval in Chinese Non-Narrative Documents
- 先用OCR提取文字，再用多模态大模型结合图像信息修正文本
- 在多个中文非叙事文档数据集上，检索效果显著优于传统方法
- 特别适合需要高精度检索的中文文档处理场景
混合检索系统结合稀疏与稠密检索方法，在处理繁体中文非叙事文档时面临挑战，主要源于复杂版式、丰富词汇及通用嵌入模型对中文同义词理解不足。现有方法未能有效兼顾两类检索需求，多聚焦于通用文本质量提升而非检索优化。本文提出知识感知预处理（KAP）框架，将噪声OCR输出转化为适配检索的高质量文本。KAP采用两阶段设计：首先通过OCR提取文本，再利用多模态大语言模型结合原始文档视觉信息进行文本精炼。该方法有效降低OCR噪声，重建结构元素，并按稀疏与稠密检索的不同要求格式化文本。实验表明，KAP在多个中文非叙事文档数据集上持续且显著优于传统预处理方法。代码已开源：https://github.com/JustinHsu1019/KAP。
原文摘要 · Abstract (English)
Hybrid Retrieval systems, combining Sparse and Dense Retrieval methods, struggle with Traditional Chinese non-narrative documents due to their complex formatting, rich vocabulary, and the insufficient understanding of Chinese synonyms by common embedding models. Previous approaches inadequately address the dual needs of these systems, focusing mainly on general text quality improvement rather than optimizing for retrieval. We propose Knowledge-Aware Preprocessing (KAP), a novel framework that transforms noisy OCR outputs into retrieval-optimized text. KAP adopts a two-stage approach: it first extracts text using OCR, then employs Multimodal Large Language Models to refine the output by integrating visual information from the original documents. This design reduces OCR noise, reconstructs structural elements, and formats the text to satisfy the distinct requirements of sparse and dense retrieval. Empirical results demonstrate that KAP consistently and significantly outperforms conventional preprocessing approaches. Our code is available at https://github.com/JustinHsu1019/KAP.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。