让检索智能回退,从失败中学习并动态权衡成本与精度。
Failure is Feedback: History-Aware Backtracking for Agentic Traversal in Multimodal Graphs
- 引入历史感知回溯机制,利用失败经验指导后续路径选择。
- 在多模态问答数据集上超越现有方法,最高提升12.3%准确率。
- 适合需要高效、自适应推理的复杂文档检索场景。
开放域多模态文档检索旨在从大规模互联文档集合中定位特定组件(如段落、表格或图像)。现有基于图的检索方法通常依赖统一相似度度量,忽略跳数相关的语义差异,且预设固定策略难以动态纠错。为此,我们提出“失败即反馈”(Failure is Feedback, FiF),将子图检索建模为序列决策过程,并引入两项创新:(i) 历史感知回溯机制,不同于传统回溯仅恢复状态,本方法利用失败路径的上下文信息,挖掘先前错误中的洞察;(ii) 经济理性代理工作流,相比静态策略的代理,其调度器采用成本感知的遍历方式,仅在前期失败证明值得时才启用高耗时的LLM推理。大量实验表明,FiF在MultimodalQA、MMCoQA和WebQA三个基准上达到当前最优性能。
原文摘要 · Abstract (English)
Open-domain multimodal document retrieval aims to retrieve specific components (paragraphs, tables, or images) from large and interconnected document corpora. Existing graph-based retrieval approaches typically rely on a uniform similarity metric that overlooks hop-specific semantics, and their rigid pre-defined plans hinder dynamic error correction. These limitations suggest that a retriever should adapt its reasoning to the evolving context and recover intelligently from dead ends. To address these needs, we propose Failure is Feedback (FiF), which casts subgraph retrieval as a sequential decision process and introduces two key innovations. (i) We introduce a history-aware backtracking mechanism; unlike standard backtracking that simply reverts the state, our approach piggybacks on the context of failed traversals, leveraging insights from previous failures. (ii) We implement an economically-rational agentic workflow. Unlike conventional agents with static strategies, our orchestrator employs a cost-aware traversal method to dynamically manage the trade-off between retrieval accuracy and inference costs, escalating to intensive LLM-based reasoning only when the prior failure justifies the additional computational investment. Extensive experiments show that FiF achieves state-of-the-art retrieval on the benchmarks of MultimodalQA, MMCoQA and WebQA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。