用知识图谱生成可验证的医学推理步骤,提升大模型诊疗能力
MedReason: Eliciting Factual Medical Reasoning Steps in LLMs via Knowledge Graphs
- 基于医学知识图谱将临床问答转为逻辑推理链
- 构建32,682条带逐步解释的医疗问答数据,准确率提升7.7%
- 经多学科医生验证,适合医疗AI可解释性研究
诊断与治疗规划等医疗任务需要精确且复杂的推理,尤其在生命攸关领域。与数学推理不同,医学推理需严谨、可验证的思维过程以保障可靠性和准确性。然而,现有缺乏提供透明、分步推理的高质量数据集来验证和增强AI模型的医学推理能力。为此,我们提出MedReason,一个大规模高质量医学推理数据集,旨在实现大语言模型(LLMs)中可信且可解释的医学问题求解。利用结构化医学知识图谱(KG),我们将临床问答对转换为逻辑推理链(即“思考路径”),追踪从问题要素到答案的相关KG实体连接。每条路径均经过临床逻辑和循证医学的一致性验证。我们的流水线从7个医疗数据集生成各类医疗问题的详细推理,共获得32,682个问答对,每个都附有详细的逐步解释。实验表明,使用该数据集微调显著提升了医疗问题求解能力,使DeepSeek-Distill-8B最高提升7.7%。表现最佳的MedReason-8B模型在临床基准MedBullets上优于当前顶尖的Huatuo-o1-8B模型达4.2%。我们还邀请来自不同专业的医疗人员评估数据质量,确保其具备准确性和连贯性。数据、模型与代码已开源:https://github.com/UCSC-VLAA/MedReason。
原文摘要 · Abstract (English)
Medical tasks such as diagnosis and treatment planning require precise and complex reasoning, particularly in life-critical domains. Unlike mathematical reasoning, medical reasoning demands meticulous, verifiable thought processes to ensure reliability and accuracy. However, there is a notable lack of datasets that provide transparent, step-by-step reasoning to validate and enhance the medical reasoning ability of AI models. To bridge this gap, we introduce MedReason, a large-scale high-quality medical reasoning dataset designed to enable faithful and explainable medical problem-solving in large language models (LLMs). We utilize a structured medical knowledge graph (KG) to convert clinical QA pairs into logical chains of reasoning, or ``thinking paths'', which trace connections from question elements to answers via relevant KG entities. Each path is validated for consistency with clinical logic and evidence-based medicine. Our pipeline generates detailed reasoning for various medical questions from 7 medical datasets, resulting in a dataset of 32,682 question-answer pairs, each with detailed, step-by-step explanations. Experiments demonstrate that fine-tuning with our dataset consistently boosts medical problem-solving capabilities, achieving significant gains of up to 7.7% for DeepSeek-Ditill-8B. Our top-performing model, MedReason-8B, outperforms the Huatuo-o1-8B, a state-of-the-art medical reasoning model, by up to 4.2% on the clinical benchmark MedBullets. We also engage medical professionals from diverse specialties to assess our dataset's quality, ensuring MedReason offers accurate and coherent medical reasoning. Our data, models, and code is available at https://github.com/UCSC-VLAA/MedReason.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。