多智能体框架提升医疗问答可信度,融合证据检索与偏见检测。
A Multi-Agent Framework for Medical AI: Leveraging Fine-Tuned GPT, LLaMA, and DeepSeek R1 for Evidence-Based and Bias-Aware Clinical Query Processing
- 分阶段使用微调的GPT、LLaMA和DeepSeek R1,分工协作提升医疗问答质量。
- 系统准确率达87%,证据增强使困惑度降至4.13,延迟36.5秒内。
- 引入不确定性评估与偏见检测,适合高风险临床决策支持场景。
大型语言模型在医疗问答中前景广阔,但受限于验证不足、证据支撑弱及置信度不可靠。本文提出一种多智能体医疗QA框架,结合互补的LLM、证据检索、不确定性估计与偏见检测以提升答案可靠性。第一阶段在包含20,000+问答对的MedQuAD衍生数据集上微调GPT、LLaMA和DeepSeek R1,DeepSeek R1在零样本评估中表现最佳(ROUGE-1 0.536 ± 0.04;ROUGE-2 0.226 ± 0.03;BLEU 0.098 ± 0.018),显著优于BioGPT。第二阶段构建模块化多智能体流程:临床推理代理(微调的LLaMA)生成结构化解释,证据检索代理查询PubMed获取最新文献依据,精炼代理(DeepSeek R1)优化表达清晰度与事实一致性;高风险或高不确定性案例可触发人工验证。安全机制包括蒙特卡洛丢弃与基于困惑度的不确定性评分,以及基于词法和情感的偏见检测,辅以LIME/SHAP分析。评估显示,全系统准确率达87%,相关性约0.80,证据增强后困惑度降为4.13,端到端平均延迟36.5秒。结果表明,智能体专业化与验证层可有效缓解单模型缺陷,提供可扩展的循证与抗偏见医疗AI方案。
原文摘要 · Abstract (English)
Large language models (LLMs) show promise for healthcare question answering, but clinical use is limited by weak verification, insufficient evidence grounding, and unreliable confidence signalling. We propose a multi-agent medical QA framework that combines complementary LLMs with evidence retrieval, uncertainty estimation, and bias checks to improve answer reliability. Our approach has two phases. First, we fine-tune three representative LLM families (GPT, LLaMA, and DeepSeek R1) on MedQuAD-derived medical QA data (20k+ question-answer pairs across multiple NIH domains) and benchmark generation quality. DeepSeek R1 achieves the strongest scores (ROUGE-1 0.536 +- 0.04; ROUGE-2 0.226 +-0.03; BLEU 0.098 -+ 0.018) and substantially outperforms the specialised biomedical baseline BioGPT in zero-shot evaluation. Second, we implement a modular multi-agent pipeline in which a Clinical Reasoning agent (fine-tuned LLaMA) produces structured explanations, an Evidence Retrieval agent queries PubMed to ground responses in recent literature, and a Refinement agent (DeepSeek R1) improves clarity and factual consistency; an optional human validation path is triggered for high-risk or high-uncertainty cases. Safety mechanisms include Monte Carlo dropout and perplexity-based uncertainty scoring, plus lexical and sentiment-based bias detection supported by LIME/SHAP-based analyses. In evaluation, the full system achieves 87% accuracy with relevance around 0.80, and evidence augmentation reduces uncertainty (perplexity 4.13) compared to base responses, with mean end-to-end latency of 36.5 seconds under the reported configuration. Overall, the results indicate that agent specialisation and verification layers can mitigate key single-model limitations and provide a practical, extensible design for evidence-based and bias-aware medical AI.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。