让检索策略自动进化,提升多模态文档推理能力
Hybrid Retriever Evolution for Multimodal Document Reasoning Agents
- 用元智能体动态调整不同检索器的使用时机与组合方式
- 在两个数据集上最高提升19.6分,超越现有主流系统
- 适合研究多步推理与智能体协作的开发者和研究人员
不同检索方法(词法、语义、多模态)在多模态文档理解中具有互补优势,但现有系统多采用固定流水线,无法适应不同推理步骤的需求。本文提出一种基于失败驱动的进化框架,让元智能体自主学习任务智能体如何在多步文档问答中协调多种检索器。元智能体分析错误推理路径,主动探测工具环境诊断根源,并迭代重写任务智能体指令,将检索从固定前端转变为动态的分步推理决策。演化后的智能体学会何时调用各检索器、如何组合以及跨模态、跨页面整合证据。在MMLongBench-Doc和DocBench上,该方法相比未演化的基线最高提升19.6点,且持续优于MACT、MDocAgent和SimpleDoc等近期系统。详细检索分析表明,性能提升源于自适应路由与证据融合,而非依赖任何硬编码检索模式;演化过程显示从单一词法行为逐步转向复杂多工具协作。这些发现确立了自主多智能体协同作为多模态文档推理的有前景范式。
原文摘要 · Abstract (English)
Different retrievers, including lexical, semantic, and multimodal approaches, provide highly complementary strengths for multimodal document understanding, yet most systems combine them through fixed pipelines that cannot adapt to the demands of individual reasoning steps. In this work, we ask whether retrieval orchestration itself can be learned as part of the reasoning process. We introduce a failure-driven evolution framework in which a meta-agent autonomously discovers how a tool-using task agent should coordinate diverse retrievers during multi-step document question answering. The meta-agent analyzes incorrect reasoning trajectories, actively probes the same tool environment to diagnose root causes, and iteratively rewrites the task agent's instructions, turning retrieval from a fixed front-end stage into an adaptive, step-wise reasoning decision. The evolved agent learns when to invoke each retriever, how to combine them, and how to compose evidence across modalities and pages. On MMLongBench-Doc and DocBench, the evolved agent achieves gains of up to +19.6 points over the unevolved baseline and consistently outperforms recent systems including MACT, MDocAgent, and SimpleDoc. Detailed retrieval analyses confirm that these improvements arise from adaptive routing and evidence composition rather than reliance on any hard coded retrieval mode, and evolution dynamics reveal a progressive shift from narrow lexical behavior to rich multi-tool coordination. These findings establish autonomous multi-agent coordination as a promising paradigm for multimodal document reasoning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。