针对长文档多模态问答，动态调整证据获取策略以提升准确率。
TAP-RAG: Task-Aware Policy Control for Long-Document Multimodal Question Answering
- 根据任务类型自动规划证据获取路径，避免固定检索方式。
- 在两个基准数据集上分别提升9.1和4.5个百分点，最高达70.2%
- 适合需要精细理解图文结构的复杂问答场景
长文档多模态问答不仅依赖从大文档中检索相关片段，不同问题还需不同类型的证据行为。现有系统通过文本块、页面图像、图链接或异构文档元素提升证据获取能力，但通常采用通用的证据使用策略。本文提出TAP-RAG框架，包含任务感知策略控制器（TAPC）以及两个策略引导的证据执行器：任务感知查询引导流扩散（TA-QFD）与任务感知视觉增强（TAVE）。TAPC为每个问题预测任务先验，评估视觉/局部/全局证据信号并生成可执行策略。TA-QFD在多模态文档图中扩展文本与结构证据，TAVE则在需要时选择性分析页面图像。受保护的融合阶段综合文本、视觉与结构证据，并在支持不足时拒绝回答。在DocBench和MMLongBench-Doc上，TAP-RAG优于对比系统，较匹配的多模态RAG基线分别提升+9.1（61.1→70.2）和+4.5（42.2→46.7）个百分点。
原文摘要 · Abstract (English)
Long-document multimodal question answering requires more than retrieving relevant chunks from a large document. Different queries require different evidence behavior. Existing multimodal RAG systems improve evidence access through text chunks, page images, graph links, or heterogeneous document elements, but they often apply a largely query-agnostic evidence-use strategy. We present TAP-RAG, a task-aware policy-controlled RAG framework for long-document multimodal QA. TAP-RAG contains a main controller, the Task-Aware Policy Controller (TAPC), and two policy-guided evidence executors: Task-Aware Query-Guided Flow Diffusion (TA-QFD) and Task-Aware Visual Enhancement (TAVE). For each query, TAPC predicts the task prior, estimates visual/local/global evidence signals, and produces an executable policy. TA-QFD then expands textual and structural evidence over the multimodal document graph, while TAVE selectively inspects page images when visual or layout evidence is needed. A guarded synthesis stage fuses text, visual, and structural evidence and abstains when support is insufficient. On DocBench and MMLongBench-Doc, TAP-RAG achieves the best overall accuracy among the compared systems, improving over a matched multimodal-RAG baseline by +9.1 points (61.1 to 70.2) and +4.5 points (42.2 to 46.7), respectively.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。