arXiv:2508.05197cs.AIcs.CL2025-08被引 5

动态融合图文检索,提升复杂视觉问答的准确率。

QA-Dragon: Query-Aware Dynamic RAG System for Knowledge-Intensive Visual Question Answering

  • 根据问题领域动态选择图文检索策略,支持多跳推理。
  • 在单源、多源和多轮任务上分别提升5.06%、6.35%、5.03%准确率。
  • 适合需要多模态知识融合的复杂视觉问答场景。

检索增强生成(RAG)通过引入外部知识缓解多模态大模型在视觉问答中的幻觉问题,已成为知识密集型视觉问答的主流方法。然而,现有RAG方法通常孤立地从文本或图像中检索,难以应对需多跳推理或最新事实知识的复杂查询。为此,我们提出QA-Dragon:一种面向知识密集型视觉问答的查询感知动态RAG系统。该系统引入领域路由机制识别查询所属领域以实现领域特定推理,并设计搜索路由动态选择最优检索策略。通过协同调度文本与图像检索代理,系统支持多模态、多轮及多跳推理,有效解决复杂视觉问答任务。我们在KDD Cup 2025的Meta CRAG-MM挑战赛上评估了该框架,显著提升了基线模型在挑战性场景下的推理性能。在单源任务上准确率提升5.06%,多源任务提升6.35%,多轮任务提升5.03%。

原文摘要 · Abstract (English)

Retrieval-Augmented Generation (RAG) has been introduced to mitigate hallucinations in Multimodal Large Language Models (MLLMs) by incorporating external knowledge into the generation process, and it has become a widely adopted approach for knowledge-intensive Visual Question Answering (VQA). However, existing RAG methods typically retrieve from either text or images in isolation, limiting their ability to address complex queries that require multi-hop reasoning or up-to-date factual knowledge. To address this limitation, we propose QA-Dragon, a Query-Aware Dynamic RAG System for Knowledge-Intensive VQA. Specifically, QA-Dragon introduces a domain router to identify the query's subject domain for domain-specific reasoning, along with a search router that dynamically selects optimal retrieval strategies. By orchestrating both text and image search agents in a hybrid setup, our system supports multimodal, multi-turn, and multi-hop reasoning, enabling it to tackle complex VQA tasks effectively. We evaluate our QA-Dragon on the Meta CRAG-MM Challenge at KDD Cup 2025, where it significantly enhances the reasoning performance of base models under challenging scenarios. Our framework achieves substantial improvements in both answer accuracy and knowledge overlap scores, outperforming baselines by 5.06% on the single-source task, 6.35% on the multi-source task, and 5.03% on the multi-turn task.

视觉问答检索增强多模态动态路由

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。