解决医学问答中的幻觉问题,让AI回答更可信。
MedTrust-RAG: Evidence Verification and Trust Alignment for Biomedical Question Answering
- 要求回答必须基于检索到的文献,无证据时用结构化否定陈述
- 通过迭代验证和问题优化,直到获得可靠信息为止
- 结合真实与错误样本训练,有效减少虚构内容
生物医学问答需要准确理解复杂的医学知识。大语言模型在该领域展现潜力,检索增强生成(RAG)系统通过引入外部医学文献提升了性能。然而,当前基于RAG的方法在生物医学问答中仍存在因后检索噪声及证据验证不足导致的幻觉问题,影响回答可靠性。本文提出MedTrust-Guided Iterative RAG框架,以提升事实一致性并缓解幻觉。方法包含三项创新:一是强制引用感知推理,所有生成内容需明确基于检索到的医学文献,证据不足时使用结构化负向知识陈述;二是采用迭代检索-验证流程,由验证代理评估证据充分性,并通过医学知识缺口分析优化查询,直至获取可靠信息;三是集成MedTrust-Align模块(MTAM),融合经验证的正例与幻觉敏感的负例样本,利用直接偏好优化强化引用基础推理,同时惩罚易产生幻觉的回应模式。
原文摘要 · Abstract (English)
Biomedical question answering (QA) requires accurate interpretation of complex medical knowledge. Large language models (LLMs) have shown promising capabilities in this domain, with retrieval-augmented generation (RAG) systems enhancing performance by incorporating external medical literature. However, RAG-based approaches in biomedical QA suffer from hallucinations due to post-retrieval noise and insufficient verification of retrieved evidence, undermining response reliability. We propose MedTrust-Guided Iterative RAG, a framework designed to enhance factual consistency and mitigate hallucinations in medical QA. Our method introduces three key innovations. First, it enforces citation-aware reasoning by requiring all generated content to be explicitly grounded in retrieved medical documents, with structured Negative Knowledge Assertions used when evidence is insufficient. Second, it employs an iterative retrieval-verification process, where a verification agent assesses evidence adequacy and refines queries through Medical Gap Analysis until reliable information is obtained. Third, it integrates the MedTrust-Align Module (MTAM) that combines verified positive examples with hallucination-aware negative samples, leveraging Direct Preference Optimization to reinforce citation-grounded reasoning while penalizing hallucination-prone response patterns.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。