用强化学习提升多跳问答的全局推理能力，仅需少量数据即达领先效果。
GlobalRAG: Enhancing Global Reasoning in Multi-hop Question Answering via Reinforcement Learning
- 将问题拆解为子目标，协同检索与推理并迭代优化证据
- 在多个数据集上以8000条训练数据实现EM/F1提升14.2%
- 适合需要高效多步推理的智能问答系统开发者
强化学习在检索增强生成（RAG）中展现潜力，但在多跳问答任务中受限于两点：缺乏全局规划以构建多步推理结构，以及执行不忠实导致查询生成与证据使用不一致。本文提出GlobalRAG，一种基于强化学习的框架，通过分解问题为子目标、协调检索与推理、迭代优化证据来增强全局推理能力。引入规划质量奖励和子目标完成奖励，引导连贯规划与可靠执行，并采用渐进式权重衰减策略平衡过程导向与结果导向目标。在域内与域外基准上大量实验表明，GlobalRAG仅使用8000条训练数据（为强基线的42%），平均在EM与F1上分别提升14.2%，显著优于现有方法。
原文摘要 · Abstract (English)
Reinforcement learning has recently shown promise in improving retrieval-augmented generation (RAG). Despite these advances, its effectiveness in multi-hop question answering (QA) remains limited by two fundamental limitations: (i) global planning absence to structure multi-step reasoning, and (ii) unfaithful execution, which hinders effective query formulation and consistent use of retrieved evidence. We propose GlobalRAG, a reinforcement learning framework designed to enhance global reasoning in multi-hop QA. GlobalRAG decomposes questions into subgoals, coordinates retrieval with reasoning, and refines evidence iteratively. To guide this process, we introduce Planning Quality Reward and SubGoal Completion Reward, which encourage coherent planning and reliable subgoal execution. In addition, a progressive weight annealing strategy balances process-oriented and outcome-based objectives. Extensive experiments on both in-domain and out-of-domain benchmarks demonstrate that GlobalRAG significantly outperforms strong baselines while using only 8k training data (42% of the training data used by strong baselines), achieving average improvements of 14.2% in both EM and F1.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。