arXiv:2411.00300cs.CL2024-11NAACL被引 76

用推理引导增强医学问答的检索生成,减少幻觉与偏见。

Rationale-Guided Retrieval Augmented Generation for Medical Question Answering

  • 通过推理文本筛选信息片段,过滤无关内容。
  • 用大模型自动生成推理作为查询,提升检索相关性。
  • 从四个医学语料库均衡取样,缓解检索器偏见。

大型语言模型(LLM)在生物医学领域有巨大潜力,但存在幻觉和知识过时问题。虽然检索增强生成(RAG)常用于解决这些问题,但仍面临三大挑战:(1)LLM易受无关或错误上下文干扰;(2)医学问题常难以精准定位有效信息;(3)检索器易受训练语料库的固有偏见影响。本文提出RAG²(Rationale-Guided RAG),一种提升生物医学场景下RAG可靠性的新框架。RAG²包含三项创新:(1)使用基于困惑度标签的小型过滤模型,仅保留文档中有信息量的片段并剔除干扰项;(2)利用大模型生成的推理作为查询,提升检索片段的实用性;(3)设计结构使检索片段均匀来自四个综合性生物医学语料库,有效缓解检索器偏见。实验表明,RAG²可提升不同规模的前沿LLM性能,最高提升6.1%;在三个医学问答基准上,相比先前最优的医疗RAG模型,性能最高提升5.6%。代码已开源:https://github.com/dmis-lab/RAG2。

原文摘要 · Abstract (English)

Large language models (LLM) hold significant potential for applications in biomedicine, but they struggle with hallucinations and outdated knowledge. While retrieval-augmented generation (RAG) is generally employed to address these issues, it also has its own set of challenges: (1) LLMs are vulnerable to irrelevant or incorrect context, (2) medical queries are often not well-targeted for helpful information, and (3) retrievers are prone to bias toward the specific source corpus they were trained on. In this study, we present RAG$^2$ (RAtionale-Guided RAG), a new framework for enhancing the reliability of RAG in biomedical contexts. RAG$^2$ incorporates three key innovations: a small filtering model trained on perplexity-based labels of rationales, which selectively augments informative snippets of documents while filtering out distractors; LLM-generated rationales as queries to improve the utility of retrieved snippets; a structure designed to retrieve snippets evenly from a comprehensive set of four biomedical corpora, effectively mitigating retriever bias. Our experiments demonstrate that RAG$^2$ improves the state-of-the-art LLMs of varying sizes, with improvements of up to 6.1\%, and it outperforms the previous best medical RAG model by up to 5.6\% across three medical question-answering benchmarks. Our code is available at https://github.com/dmis-lab/RAG2.

医学问答检索增强大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。