让大模型同时理解文档中的文字和表格,提升复杂文档问答效果。
Mixture-of-RAG: Integrating Text and Tables with Large Language Models
- 分层行列建模保留文本与表格的结构关系
- 检索精度比单一文本/表格方法高46%
- 适合处理年报、论文等含图文混合内容的场景
大语言模型在响应时若能基于外部知识源,表现更佳。然而真实文档如年报、科研论文和临床指南常包含大量叙述性文字与复杂层级表格。现有检索增强生成(RAG)系统在处理这类异构文本-表格结构时性能显著下降。为此,我们提出异构文档RAG任务,并设计MixRAG三阶段框架:(i) 层级行列级(H-RCL)表示以保留层级结构与异构关系;(ii) 基于LLM重排序的集成检索器实现证据对齐;(iii) 通过RECAP提示策略进行多步推理分解。为填补数据空白,我们发布DocRAGLib数据集,包含2,000份文档及其自动生成的文表摘要与人工标注的黄金标准。实验表明,MixRAG在顶级检索准确率上相比强基线(纯文本、纯表格、简单融合)提升46%,确立了混合模态文档接地的新基准。
原文摘要 · Abstract (English)
Large language models (LLMs) achieve optimal utility when their responses are grounded in external knowledge sources. However, real-world documents, such as annual reports, scientific papers, and clinical guidelines, frequently combine extensive narrative content with complex, hierarchically structured tables. While existing retrieval-augmented generation (RAG) systems effectively integrate LLMs' generative capabilities with external retrieval-based information, their performance significantly deteriorates especially processing such heterogeneous text-table hierarchies. To address this limitation, we formalize the task of Heterogeneous Document RAG, which requires joint retrieval and reasoning across textual and hierarchical tabular data. We propose MixRAG, a novel three-stage framework: (i) hierarchy row-and-column-level (H-RCL) representation that preserves hierarchical structure and heterogeneous relationship, (ii) an ensemble retriever with LLM-based reranking for evidence alignment, and (iii) multi-step reasoning decomposition via a RECAP prompt strategy. To bridge the gap in available data for this domain, we release the dataset DocRAGLib, a 2k-document corpus paired with automatically aligned text-table summaries and gold document annotations. The comprehensive experiment results demonstrate that MixRAG boosts top-1 retrieval by 46% over strong text-only, table-only, and naive-mixture baselines, establishing new state-of-the-art performance for mixed-modality document grounding.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。