arXiv:2510.25160cs.CLcs.AI2025-10被引 2

让大模型更高效地理解文档,通过结构化知识转化提升搜索准确率。

Model-Document Protocol for AI Search

  • 将原始文档转化为适合大模型使用的结构化知识,而非直接返回碎片文本。
  • 在信息检索任务中显著超越基线,提升关键指标表现。
  • 适合需要精准理解长文本的智能搜索、问答系统研发者。

AI 搜索依赖于将大语言模型(LLMs)与海量外部知识源连接。然而,网页、PDF 等原始文档本身并非 LLM 友好:内容冗长、噪声多、无结构。传统检索方法将文档视为原始文本,返回片段,迫使大模型自行拼合和推理。这凸显了亟需一种新的检索范式。本文提出模型-文档协议(MDP),一个通用框架,规范原始文本如何通过可消费的知识表示被转化为大模型可用的形式。MDP 定义了三种路径:代理推理(将原始证据整理为连贯上下文)、记忆锚定(积累可复用笔记以增强推理)、结构化利用(将文档编码为图或键值缓存等形式)。三者共同目标是确保传递给大模型的不是原始片段,而是紧凑、结构化的可推理知识。作为实例,我们提出 MDP-Agent,通过代理流程实现该协议:构建文档级概要记忆以实现全局覆盖,采用基于扩散的探索与垂直挖掘发现深层依赖,使用地图-归约式合成将大规模证据整合为紧凑但充分的上下文。在信息获取基准测试中,MDP-Agent 显著优于基线,验证了 MDP 框架的合理性及其代理实例的有效性。

原文摘要 · Abstract (English)

AI search depends on linking large language models (LLMs) with vast external knowledge sources. Yet web pages, PDF files, and other raw documents are not inherently LLM-ready: they are long, noisy, and unstructured. Conventional retrieval methods treat these documents as verbatim text and return raw passages, leaving the burden of fragment assembly and contextual reasoning to the LLM. This gap underscores the need for a new retrieval paradigm that redefines how models interact with documents. We introduce the Model-Document Protocol (MDP), a general framework that formalizes how raw text is bridged to LLMs through consumable knowledge representations. Rather than treating retrieval as passage fetching, MDP defines multiple pathways that transform unstructured documents into task-specific, LLM-ready inputs. These include agentic reasoning, which curates raw evidence into coherent context; memory grounding, which accumulates reusable notes to enrich reasoning; and structured leveraging, which encodes documents into formal representations such as graphs or key-value caches. All three pathways share the same goal: ensuring that what reaches the LLM is not raw fragments but compact, structured knowledge directly consumable for reasoning. As an instantiation, we present MDP-Agent, which realizes the protocol through an agentic process: constructing document-level gist memories for global coverage, performing diffusion-based exploration with vertical exploitation to uncover layered dependencies, and applying map-reduce style synthesis to integrate large-scale evidence into compact yet sufficient context. Experiments on information-seeking benchmarks demonstrate that MDP-Agent outperforms baselines, validating both the soundness of the MDP framework and the effectiveness of its agentic instantiation.

AI搜索文档理解大模型知识推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。