让知识图谱像游戏一样自我进化,提升多模态问答准确率
EvoGraph-R1: Self-Evolving Multimodal Knowledge Hypergraphs for Agentic Retrieval

- 把知识图谱变成可交互的动态环境,用智能体逐步优化检索
- 在多模态VQA和文本问答上,准确率、覆盖范围均超越现有基线
- 适合需要持续更新、多轮推理的智能系统研发者
检索增强生成(RAG)已成为将多模态大模型(MLLMs)与外部知识结合的关键范式。现有GraphRAG方法引入结构化实体-关系图以提升检索与推理能力,但受限于将知识图谱视为离线构建的静态数据结构,仅支持单次查询。这种静态模式与知识密集型推理的交互性、迭代性不匹配,存在三大瓶颈:(i) 以文本为中心的碎片化阻碍跨模态推理,(ii) 固定结构无法融入新证据或修正错误,(iii) 机械的单次检索缺乏自适应优化。为此,我们提出EvoGraph-R1,一种自演化GraphRAG框架,将知识图谱重新构想为通过智能体交互不断演化的动态环境。我们将检索建模为马尔可夫决策过程(MDP),智能体观察图状态并执行动作:查询(GraphRetrieve)、扩展(WebSearch)、精炼(GraphEdit)或终止(Answer),这些动作重塑超图结构并生成反馈信号以指导后续演化。通过这一闭环机制,超图持续整合新证据、修正错误、优化结构,支持多跳推理。在多模态VQA与文本QA基准上的实验表明,相比现有RAG基线,EvoGraph-R1在准确率、覆盖率与可追溯性方面均有显著提升,确立了自演化知识图谱作为跨模态基础范式的潜力。
原文摘要 · Abstract (English)
Retrieval-augmented generation (RAG) has emerged as a critical paradigm for grounding Multimodal Large Language Models (MLLMs) in external knowledge. Recent GraphRAG methods introduce structured entity-relation graphs to improve retrieval and reasoning. However, they remain limited by treating knowledge graphs as static data structures built offline and queried in a single pass. This static paradigm misaligns with the interactive, iterative nature of knowledge-intensive reasoning, creating three bottlenecks: (i) text-centric fragmentation that impedes cross-modal reasoning, (ii) frozen structures unable to incorporate new evidence or correct errors, and (iii) rigid single-pass retrieval without adaptive refinement. To overcome these limitations, we introduce EvoGraph-R1, a self-evolving GraphRAG framework that reconceptualizes knowledge graphs as dynamic environments shaped through agent interactions. We formulate retrieval as a Markov Decision Process (MDP) where the agent observes the graph state and executes actions to query (GraphRetrieve), expand (WebSearch), refine (GraphEdit), or terminate (Answer) the reasoning. These actions reshape the hypergraph structure and generate feedback signals that guide subsequent evolution. Through this closed loop, the hypergraph evolves by integrating new evidence, correcting errors, and refining structure to support multi-hop reasoning. Experiments on multimodal VQA and text QA benchmarks demonstrate substantial improvements over existing RAG baselines in accuracy, coverage, and traceability, establishing self-evolving knowledge graphs as a fundamental paradigm across modalities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。