用检索增强提升医疗问答准确率,减少幻觉。
Optimizing Medical Question-Answering Systems: A Comparative Study of Fine-Tuned and Zero-Shot Large Language Models with RAG Framework
- 结合医学文献检索与开源大模型生成答案
- 微调后模型在PubMedQA上准确率达71.8%,比零样本提升16.4个百分点
- 通过引用来源实现透明可追溯,适合临床辅助决策场景
医疗问答系统可受益于大语言模型(LLM)的发展,但直接应用于临床领域存在事实准确性难以保证、易产生幻觉等问题。本文提出一种基于检索增强生成(RAG)的医疗问答系统,通过融合领域知识检索与开源大模型来回答医学问题。我们采用低秩适应(LoRA)对两款先进开源模型(LLaMA~2 和 Falcon)进行微调,实现高效领域适配。系统从医学文献中检索相关证据以支撑答案生成,显著提升事实正确性并减少幻觉。在PubMedQA和MedMCQA两个基准数据集上的评估表明,引入检索机制后答案准确率明显优于仅使用LLM的基线。其中,微调后的LLaMA~2模型在PubMedQA上达到71.8%准确率,较55.4%的零样本基线大幅提升;同时通过提供来源引用保持了结果透明性。分析还显示,该方法使未经支持的内容减少约60%。这些结果证明了基于RAG的开源大模型在可靠生物医学问答中的潜力,为临床信息学应用提供了可行路径。
原文摘要 · Abstract (English)
Medical question-answering (QA) systems can benefit from advances in large language models (LLMs), but directly applying LLMs to the clinical domain poses challenges such as maintaining factual accuracy and avoiding hallucinations. In this paper, we present a retrieval-augmented generation (RAG) based medical QA system that combines domain-specific knowledge retrieval with open-source LLMs to answer medical questions. We fine-tune two state-of-the-art open LLMs (LLaMA~2 and Falcon) using Low-Rank Adaptation (LoRA) for efficient domain specialization. The system retrieves relevant medical literature to ground the LLM's answers, thereby improving factual correctness and reducing hallucinations. We evaluate the approach on benchmark datasets (PubMedQA and MedMCQA) and show that retrieval augmentation yields measurable improvements in answer accuracy compared to using LLMs alone. Our fine-tuned LLaMA~2 model achieves 71.8% accuracy on PubMedQA, substantially improving over the 55.4% zero-shot baseline, while maintaining transparency by providing source references. We also detail the system design and fine-tuning methodology, demonstrating that grounding answers in retrieved evidence reduces unsupported content by approximately 60%. These results highlight the potential of RAG-augmented open-source LLMs for reliable biomedical QA, pointing toward practical clinical informatics applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。