用专家混合机制提升图结构检索增强生成的精准度
MoG: Mixture of Experts for Graph-based Retrieval-Augmented Generation
- 构建枢纽图与专家图双组件，分层组织知识
- 查询时仅激活少量专家图，减少无关信息干扰
- 在MuSiQue上相对基线提升超20%，适合复杂推理场景
检索增强生成通过外部证据提升大模型的可信性，但统一知识库易引入无关信息，误导复杂推理。受混合专家（MoE）中路由选择机制启发，我们提出图结构检索增强生成的混合专家方法（MoG）。该方法将知识分为两类核心组件：(i) 多样化、始终可访问的枢纽图，编码语义和结构上的核心知识，为专家激活提供上下文线索；(ii) 稀疏激活的专家图，包含领域特定证据。MoG首先访问枢纽图获取通用证据并生成上下文线索，随后由拓扑感知路由模块根据查询动态激活有限数量的专家图，从而将检索限定在聚焦的证据子空间内。在多个挑战性基准测试中，MoG持续优于强基线，在MuSiQue上实现超过20%的相对提升。代码已开源。
原文摘要 · Abstract (English)
Retrieval-augmented generation is intensively studied to ground large language models on external evidence. However, retrieving from a unified knowledge base could inevitably introduce irrelevant information that may mislead generation for complex reasoning. Inspired by the conditional computation of mixture of experts (MoE), where a router sparsely selects specialized experts alongside shared ones for each input, we propose \textbf{M}ixture \textbf{o}f experts for \textbf{G}raph-based Retrieval-Augmented Generation, i.e., \textbf{MoG}. It organizes knowledge into two core components: (i) diverse, always-accessible hub graphs that encode semantically and structurally central knowledge and provide contextual clues for expert activation, and (ii) sparsely activated expert graphs that contain domain-specific evidence. MoG first accesses hub graphs to identify general evidence and derive contextual clues. Then, a topology-aware router dynamically activates a limited set of expert graphs conditioned on the query, thereby confining retrieval to a focused evidence subspace. Extensive experiments on challenging benchmarks show that MoG consistently outperforms strong baselines, with over 20\% relative improvement on MuSiQue. Our code is available in https://github.com/DEEP-PolyU/MoG.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。