用视觉理解优化文档分块，让RAG更懂复杂文档结构。
Vision-Guided Chunking Is All You Need: Enhancing RAG with Multimodal Document Understanding
- 利用大模型批量处理带图文的PDF，保持跨页上下文连贯。
- 在多页表格和嵌入式图表上表现更优，提升下游问答准确率。
- 适合需要理解复杂学术或技术文档的场景，如论文检索。
检索增强生成（RAG）系统革新了信息检索与问答能力，但传统文本分块方法难以应对复杂文档结构、跨页表格、嵌入式图表及跨页上下文依赖。本文提出一种新型多模态文档分块方法，利用大语言多模态模型（LMM）批量处理PDF文档，通过可配置的页面批次与跨批次上下文保留机制，实现对跨页表格、嵌入图像和流程化内容的精准处理。我们在一个手工构建的PDF文档数据集上评估该方法，结果显示其分块质量与下游RAG性能均优于传统RAG系统。定性分析表明，该方法显著提升了文档结构与语义连贯性的保留能力。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) systems have revolutionized information retrieval and question answering, but traditional text-based chunking methods struggle with complex document structures, multi-page tables, embedded figures, and contextual dependencies across page boundaries. We present a novel multimodal document chunking approach that leverages Large Multimodal Models (LMMs) to process PDF documents in batches while maintaining semantic coherence and structural integrity. Our method processes documents in configurable page batches with cross-batch context preservation, enabling accurate handling of tables spanning multiple pages, embedded visual elements, and procedural content. We evaluate our approach on a curated dataset of PDF documents with manually crafted queries, demonstrating improvements in chunk quality and downstream RAG performance. Our vision-guided approach achieves better accuracy compared to traditional vanilla RAG systems, with qualitative analysis showing superior preservation of document structure and semantic coherence.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。