arXiv:2601.20221cs.AIcs.CL2026-01被引 2

让AI医生推理过程可验证,自动查证并自我改进。

Scaling Medical Reasoning Verification via Tool-Integrated Reinforcement Learning

  • 让AI一步步查医学资料,动态修正推理过程。
  • 在多个测试中准确率提升超20%,采样成本降为原来的1/8。
  • 适合需要高可信度的医疗AI系统开发者使用。

大型语言模型在医学推理基准上表现优异,但临床部署需确保事实准确性。现有奖励模型方法存在两大缺陷:仅输出单一评分而无解释依据,且依赖单次检索,无法在验证过程中动态获取知识。本文提出$ ext{method}$,一种基于工具增强的智能体框架,通过训练医学推理验证器在评估时迭代查询外部医学文献库来解决上述问题。该方法结合工具增强的验证与迭代强化学习范式,仅需逐条推理轨迹的监督,并引入自适应课程机制动态调整训练数据分布。在四个医学推理基准上,$ ext{method}$显著优于现有方法,相较于基础生成器,MedQA准确率提升23.5%,MedXpertQA提升32.0%。关键的是,相比先前奖励模型基线,$ ext{method}$的采样预算需求降低8倍。结果表明,将验证过程建立在动态检索证据基础上,是构建更可靠医学推理系统的一条有效路径。

原文摘要 · Abstract (English)

Large language models have achieved strong performance on medical reasoning benchmarks, yet their deployment in clinical settings demands rigorous verification to ensure factual accuracy. While reward models offer a scalable approach for reasoning trace verification, existing methods face two limitations: they produce only scalar reward values without explicit justification, and they rely on single-pass retrieval that precludes adaptive knowledge access as verification unfolds. We introduce $\method$, an agentic framework that addresses these limitations by training medical reasoning verifiers to iteratively query external medical corpora during evaluation. Our approach combines tool-augmented verification with an iterative reinforcement learning paradigm that requires only trace-level supervision, alongside an adaptive curriculum mechanism that dynamically adjusts training data distribution. Across four medical reasoning benchmarks, $\method$ achieves substantial gains over existing methods, improving MedQA accuracy by 23.5% and MedXpertQA by 32.0% relative to the base generator in particular. Crucially, $\method$ demonstrates an $\mathbf{8\times}$ reduction in sampling budget requirement compared to prior reward model baselines. These findings establish that grounding verification in dynamically retrieved evidence offers a principled path toward more reliable medical reasoning systems.

医学推理强化学习验证框架工具调用

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。