检测医学生成回答中的错误断言，提升AI医疗问答的安全性。
MedRAGChecker: Claim-Level Verification for Biomedical Retrieval-Augmented Generation
- 将回答拆解为独立断言，结合知识图谱与推理模型验证支持度。
- 在4个数据集上准确识别不支持和自相矛盾的断言，发现不同模型风险差异。
- 专为医学RAG设计，适合关注AI医疗安全的研究者与开发者。
生物医学检索增强生成（RAG）可使大模型的回答基于医学文献，但长篇输出常包含孤立且无支持或相互矛盾的断言，存在安全隐患。我们提出MedRAGChecker，一种针对生物医学RAG的断言级验证与诊断框架。给定问题、检索证据和生成答案，MedRAGChecker将答案分解为原子断言，并通过融合证据驱动的自然语言推理（NLI）与生物医学知识图谱（KG）一致性信号来评估断言支持度。聚合断言决策结果生成答案级诊断，可区分检索与生成失败类型，包括忠实性、证据不足、矛盾及关键安全错误率。为实现可扩展评估，我们提炼出轻量级生物医学模型，并采用类别特定可靠性加权的集成验证器。在四个生物医学问答基准上的实验表明，MedRAGChecker能可靠标记不支持和矛盾断言，并揭示生成器在关键生物医学关系上的不同风险特征。
原文摘要 · Abstract (English)
Biomedical retrieval-augmented generation (RAG) can ground LLM answers in medical literature, yet long-form outputs often contain isolated unsupported or contradictory claims with safety implications. We introduce MedRAGChecker, a claim-level verification and diagnostic framework for biomedical RAG. Given a question, retrieved evidence, and a generated answer, MedRAGChecker decomposes the answer into atomic claims and estimates claim support by combining evidence-grounded natural language inference (NLI) with biomedical knowledge-graph (KG) consistency signals. Aggregating claim decisions yields answer-level diagnostics that help disentangle retrieval and generation failures, including faithfulness, under-evidence, contradiction, and safety-critical error rates. To enable scalable evaluation, we distill the pipeline into compact biomedical models and use an ensemble verifier with class-specific reliability weighting. Experiments on four biomedical QA benchmarks show that MedRAGChecker reliably flags unsupported and contradicted claims and reveals distinct risk profiles across generators, particularly on safety-critical biomedical relations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。