构建结构化文档地图,让AI理解文档的层级与关联关系。
DMAP: Human-Aligned Structural Document Map for Multimodal Document Understanding
- 用人类认知对齐的层次结构组织文档内容与图文元素。
- 在MMDocQA上提升检索精度与推理一致性,优于传统RAG方法。
- 适合需要深度理解复杂文档的场景,如学术论文、报告分析。
现有多模态文档问答系统主要依赖扁平语义检索,将文档拆分为孤立文本块,忽略其内在的层级与关联结构。这种扁平化处理破坏了逻辑与空间依赖关系,如章节组织、图文对应和交叉引用,而这些正是人类理解文档的关键。为此,我们提出文档级结构化文档地图(DMAP),显式编码多模态文档中的层级结构与元素间关系。具体地,设计结构化语义理解智能体,将文本、图表、表格等内容整合为符合人类认知的层次化架构,捕捉语义与版面依赖。在此基础上,反思性推理智能体实现结构感知、证据驱动的推理,动态评估上下文充分性,并通过与DMAP的迭代交互优化答案。在MMDocQA基准上的大量实验表明,DMAP生成的结构表示更契合人类解读模式,显著提升检索精度、推理一致性和多模态理解能力,优于传统RAG方法。代码已开源。
原文摘要 · Abstract (English)
Existing multimodal document question-answering (QA) systems predominantly rely on flat semantic retrieval, representing documents as a set of disconnected text chunks and largely neglecting their intrinsic hierarchical and relational structures. Such flattening disrupts logical and spatial dependencies - such as section organization, figure-text correspondence, and cross-reference relations, that humans naturally exploit for comprehension. To address this limitation, we introduce a document-level structural Document MAP (DMAP), which explicitly encodes both hierarchical organization and inter-element relationships within multimodal documents. Specifically, we design a Structured-Semantic Understanding Agent to construct DMAP by organizing textual content together with figures, tables, charts, etc. into a human-aligned hierarchical schema that captures both semantic and layout dependencies. Building upon this representation, a Reflective Reasoning Agent performs structure-aware and evidence-driven reasoning, dynamically assessing the sufficiency of retrieved context and iteratively refining answers through targeted interactions with DMAP. Extensive experiments on MMDocQA benchmarks demonstrate that DMAP yields document-specific structural representations aligned with human interpretive patterns, substantially enhancing retrieval precision, reasoning consistency, and multimodal comprehension over conventional RAG-based approaches. Code is available at https://github.com/Forlorin/DMAP
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。