arXiv:2506.21594cs.CL2025-06被引 2

320亿参数模型用两阶段训练,医学推理能力超大模型。

Gazal-R1: Achieving State-of-the-Art Medical Reasoning with Parameter-Efficient Two-Stage Training

  • 分两阶段训练:先用合成数据教结构化思维,再用强化学习优化推理质量。
  • 在MedQA等三基准上分别达87.1%、81.6%、79.6%,超越最大12倍大的模型。
  • 适合医疗AI研发者,兼具高准确率与可解释性,方法可复现。

我们提出Gazal-R1,一个320亿参数的语言模型,在医学推理任务中达到当前最优性能,并能提供透明、分步的临床决策解释。基于Qwen3 32B,该模型证明通过策略性训练,中等规模模型可在专业领域超越显著更大的模型。我们设计了新颖的两阶段训练流程:第一阶段,在107,033个精心构建的合成医学推理样本上进行监督微调,采用权重分解低秩适配(DoRA)和秩稳定LoRA(rsLoRA)等参数高效技术;第二阶段,使用组相对策略优化(GRPO)与多组件奖励系统进行强化学习,提升准确性、格式遵循度和推理质量。Gazal-R1在多个医学基准上表现优异,于MedQA得分为87.1%,MMLU Pro(Medical)为81.6%,PubMedQA为79.6%,超越参数量高达12倍的模型。除优异实证结果外,本工作还深入探讨了特定领域推理模型训练中的挑战,包括奖励劫持、训练不稳定性,以及事实记忆与详细推理间的根本矛盾。其方法为开发高性能、高效且可解释的领域专用语言模型提供了可复现框架。

原文摘要 · Abstract (English)

We present Gazal-R1, a 32-billion-parameter language model that achieves state-of-the-art performance in medical reasoning while providing transparent, step-by-step explanations for clinical decision-making. Built upon Qwen3 32B, our model demonstrates that strategic training can enable mid-sized models to outperform significantly larger counterparts in specialized domains. We developed a novel two-stage training pipeline: first, supervised fine-tuning on a carefully curated dataset of 107,033 synthetic medical reasoning examples that teaches structured clinical thinking, enhanced by advanced parameter-efficient techniques including Weight-Decomposed Low-Rank Adaptation (DoRA) and Rank-Stabilized LoRA (rsLoRA); second, reinforcement learning using Group Relative Policy Optimization (GRPO) with a sophisticated multi-component reward system that refines accuracy, format adherence, and reasoning quality. Gazal-R1 achieves exceptional performance across medical benchmarks, scoring 87.1% on MedQA, 81.6% on MMLU Pro (Medical), and 79.6% on PubMedQA, surpassing models up to 12x larger. Beyond its strong empirical results, this work provides detailed insights into the challenges of training reasoning-capable models in specialized domains, including issues with reward hacking, training instability, and the fundamental tension between factual recall and detailed reasoning. Our methodology offers a reproducible framework for developing high-capability, domain-specific language models that balance performance, efficiency, and explainability.

医学推理参数高效两阶段训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。