arXiv:2508.13579cs.AI2025-08AAAI被引 6

用专家注意力引导强化学习,提升大模型对电子病历的推理能力

Toward Better EHR Reasoning in LLMs: Reinforcement Learning with Expert Attention Guidance

  • 通过专家模型指导蒙特卡洛树搜索生成高质量推理路径
  • 使大模型注意力聚焦临床关键特征,整体推理能力提升14.62%
  • 适合需要高可靠性医疗预测的AI系统研发人员

提升大语言模型(LLM)在电子健康记录(EHR)推理中的表现对实现准确、可泛化的临床预测至关重要。尽管LLM在医学文本理解方面表现出色,但在处理时间结构化、高维的EHR数据时仍存在不足。现有方法多采用混合范式,将LLM仅作为固定的先验检索器,下游深度学习(DL)模型负责预测,未能提升LLM自身的推理能力,且继承了DL模型的泛化局限。为此,我们提出EAG-RL,一种两阶段训练框架,通过专家注意力引导,内在增强LLM的EHR推理能力。具体而言,EAG-RL首先利用专家指导的蒙特卡洛树搜索构建高质量、分步的推理轨迹,有效初始化LLM策略;随后通过强化学习进一步优化策略,使其注意力与专家EHR模型识别出的临床显著特征对齐。在两个真实世界EHR数据集上的实验表明,EAG-RL平均提升LLM内在EHR推理能力14.62%,同时增强对特征扰动的鲁棒性,并提升在未见临床领域的泛化能力。结果验证了EAG-RL在临床预测任务中实际部署的潜力。代码已开源:https://github.com/devilran6/EAG-RL。

原文摘要 · Abstract (English)

Improving large language models (LLMs) for electronic health record (EHR) reasoning is essential for enabling accurate and generalizable clinical predictions. While LLMs excel at medical text understanding, they underperform on EHR-based prediction tasks due to challenges in modeling temporally structured, high-dimensional data. Existing approaches often rely on hybrid paradigms, where LLMs serve merely as frozen prior retrievers while downstream deep learning (DL) models handle prediction, failing to improve the LLM's intrinsic reasoning capacity and inheriting the generalization limitations of DL models. To this end, we propose EAG-RL, a novel two-stage training framework designed to intrinsically enhance LLMs' EHR reasoning ability through expert attention guidance, where expert EHR models refer to task-specific DL models trained on EHR data. Concretely, EAG-RL first constructs high-quality, stepwise reasoning trajectories using expert-guided Monte Carlo Tree Search to effectively initialize the LLM's policy. Then, EAG-RL further optimizes the policy via reinforcement learning by aligning the LLM's attention with clinically salient features identified by expert EHR models. Extensive experiments on two real-world EHR datasets show that EAG-RL improves the intrinsic EHR reasoning ability of LLMs by an average of 14.62%, while also enhancing robustness to feature perturbations and generalization to unseen clinical domains. These results demonstrate the practical potential of EAG-RL for real-world deployment in clinical prediction tasks. Our code have been available at https://github.com/devilran6/EAG-RL.

医疗AI大模型强化学习电子病历

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。