arXiv:2507.23541cs.CL2025-07被引 1

用渐进强化学习提升大模型医疗推理与检索的协同能力

Med-R$^3$: Enhancing Medical Retrieval-Augmented Reasoning of LLMs via Progressive Reinforcement Learning

  • 分三阶段优化:先强逻辑推理,再调检索,最后联合协调
  • 在医疗问答上超越GPT-4o-mini 12.22%,Qwen2.5-14B提升16.31%
  • 专为医疗设计奖励函数,避免记忆固定解法,适合医学AI研究者

在医疗场景中,有效获取外部知识并用于严谨逻辑推理至关重要。现有工作多孤立提升检索或推理能力,缺乏联合优化,导致两环节协作不足。同时,依赖监督微调易使模型记忆已有解题路径,限制其对新问题的泛化能力。尽管部分研究尝试用强化学习改进通用领域检索增强推理,但其奖励函数未能充分反映医疗领域的特殊需求。为此,我们提出Med-R$^3$,一种基于渐进强化学习的医疗检索增强推理框架。首先训练模型进行医疗问题的逻辑推理;在此基础上,自适应优化检索能力,使其更契合知识库特征和推理过程中的信息利用;最后实现检索与推理的联合优化。大量实验表明,Med-R$^3$达到领先性能:Qwen3-8B + Med-R$^3$ 在参数量相当情况下,超越闭源GPT-4o-mini 12.22%;Qwen2.5-14B + Med-R$^3$ 更取得16.31%的显著提升。

原文摘要 · Abstract (English)

In medical scenarios, effectively retrieving external knowledge and leveraging it for rigorous logical reasoning is of significant importance. Despite their potential, existing work has predominantly focused on enhancing either retrieval or reasoning capabilities of the models in isolation, with little attention given to their joint optimization, which leads to limited coordination between the two processes. Additionally, current methods rely heavily on supervised fine-tuning (SFT), which can cause models to memorize existing problem-solving pathways, thereby restricting their generalization ability when confronted with novel problem contexts. Furthermore, while some studies have explored to improve retrieval-augmented reasoning in general domains via reinforcement learning, their reward function designs do not adequately capture the specific demands of the medical domain. To address these challenges, we introduce **Med-R$^3$**, a **Med**ical **R**etrieval-augmented **R**easoning framework driven by progressive **R**einforcement learning. In this framework, we first develop the model's ability to perform logical reasoning over medical problems. Subsequently, on the basis of this foundation, we adaptively optimize the retrieval capability to better align with the characteristics of knowledge corpus and external information utilization throughout the reasoning process. Finally, we conduct joint optimization of the model's retrieval and reasoning coordination. Extensive experiments indicate that **Med-R$^3$** could achieve state-of-the-art performances, with Qwen3-8B + Med-R$^3$ surpassing closed-sourced GPT-4o-mini by 12.22% at a comparable parameter scale, while Qwen2.5-14B augmented with \ours shows a more substantial gain of 16.31%.

医疗AI强化学习检索增强大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。