arXiv:2510.27261cs.CV2025-10AAAI被引 8

将文档检索细化到视觉区域,提升图文理解精度与效率。

RegionRAG: Region-level Retrieval-Augmented Generation for Visual Document Understanding

  • 以视觉区域为单位检索,精准定位查询相关图像块。
  • 平均检索准确率提升10.02%,问答准确率提高3.56%。
  • 适合需要高精度图文理解的场景,如智能客服、文档自动化。

多模态检索增强生成(RAG)已成为通过候选视觉文档赋能大语言模型的关键方法。然而,现有方法将整个文档视为基本检索单元,引入大量无关内容:一方面,相关文档常包含与查询无关的大面积区域,稀释关键信息;另一方面,为提高召回率而检索多份文档,进一步引入冗余和无关内容。这些冗余上下文分散模型注意力,降低性能。为此,我们提出RegionRAG,一种将检索范式从文档级转向区域级的新框架。训练阶段,采用标注数据与未标注数据结合的混合监督策略,精确定位相关图像块;推理阶段,设计动态流程,智能将显著图像块合并为完整语义区域。通过让检索器承担识别相关区域的任务,生成器可专注处理简洁、与查询相关的视觉内容,从而提升效率与准确性。在六个基准上的实验表明,RegionRAG达到先进水平:平均R@1检索准确率提升10.02%,问答准确率提升3.56%,同时仅使用71.42%的视觉标记数。

原文摘要 · Abstract (English)

Multi-modal Retrieval-Augmented Generation (RAG) has become a critical method for empowering LLMs by leveraging candidate visual documents. However, current methods consider the entire document as the basic retrieval unit, introducing substantial irrelevant visual content in two ways: 1) Relevant documents often contain large regions unrelated to the query, diluting the focus on salient information; 2) Retrieving multiple documents to increase recall further introduces redundant and irrelevant documents. These redundant contexts distract the model's attention and further degrade the performance. To address this challenge, we propose RegionRAG, a novel framework that shifts the retrieval paradigm from the document level to the region level. During training, we design a hybrid supervision strategy from both labeled data and unlabeled data to pinpoint relevant patches. During inference, we propose a dynamic pipeline that intelligently groups salient patches into complete semantic regions. By delegating the task of identifying relevant regions to the retriever, RegionRAG enables the generator to focus solely on concise, query-relevant visual content, improving both efficiency and accuracy. Experiments on six benchmarks demonstrate that RegionRAG achieves state-of-the-art performance. It improves retrieval accuracy by 10.02% in R@1 on average, and boosts question answering accuracy by 3.56% while using only 71.42% visual tokens compared with prior methods.

视觉理解RAG图文生成区域检索

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。