用580万医学问答对提升大模型医疗准确性,减少幻觉。
MIRIAD: Augmenting LLMs with millions of medical query-response pairs
- 构建半自动流程生成高质量医学问答对,覆盖56个医学领域。
- 相比传统RAG,医学问答准确率提升6.7%,幻觉检测能力提高22.5%~37%。
- 配套可视化地图工具,支持临床用户交互式探索知识。
大语言模型有望通过智能决策支持和灵活聊天助手重塑医疗。然而,其易产生不准确的医学内容。为使模型基于高质量医学知识,现有方法采用检索增强生成(RAG),将非结构化医学文本切分为小片段以检索整合。但此类文本常含噪声、未清洗,难以有效利用。为此,我们提出MIRIAD,一个包含5,821,948条医学问答对的大规模、结构化语料库,每条问答均源自同行评审文献并经半自动化流程生成、筛选、验证与人工标注。不同于以往依赖非结构化文本的语料库,MIRIAD以可检索的问答形式组织海量医学知识,实现更精准的召回。在挑战性医学问答基准测试中,使用MIRIAD增强的模型相比相同来源语料的非结构化RAG基线,准确率最高提升6.7%,且在检测医学幻觉方面,F1分数提升22.5%至37%。我们进一步推出MIRIAD-Atlas——一个涵盖56个医学领域的交互式知识地图,支持临床用户可视化浏览、搜索与精炼知识。MIRIAD可推动医疗信息检索器、增强型RAG应用及知识驱动聊天界面等下游应用,助力医疗领域更可靠的LLM落地。
原文摘要 · Abstract (English)
LLMs are bound to transform healthcare with advanced decision support and flexible chat assistants. However, LLMs are prone to generate inaccurate medical content. To ground LLMs in high-quality medical knowledge, LLMs have been equipped with external knowledge via RAG, where unstructured medical knowledge is split into small text chunks that can be selectively retrieved and integrated into the LLMs context. Yet, existing RAG pipelines rely on raw, unstructured medical text, which can be noisy, uncurated and difficult for LLMs to effectively leverage. Systematic approaches to organize medical knowledge to best surface it to LLMs are generally lacking. To address these challenges, we introduce MIRIAD, a large-scale, curated corpus of 5,821,948 medical QA pairs, each rephrased from and grounded in a passage from peer-reviewed medical literature using a semi-automated pipeline combining LLM generation, filtering, grounding, and human annotation. Unlike prior medical corpora, which rely on unstructured text, MIRIAD encapsulates web-scale medical knowledge in an operationalized query-response format, which enables more targeted retrieval. Experiments on challenging medical QA benchmarks show that augmenting LLMs with MIRIAD improves accuracy up to 6.7% compared to unstructured RAG baselines with the same source corpus and with the same amount of retrieved text. Moreover, MIRIAD improved the ability of LLMs to detect medical hallucinations by 22.5 to 37% (increase in F1 score). We further introduce MIRIAD-Atlas, an interactive map of MIRIAD spanning 56 medical disciplines, enabling clinical users to visually explore, search, and refine medical knowledge. MIRIAD promises to unlock a wealth of down-stream applications, including medical information retrievers, enhanced RAG applications, and knowledge-grounded chat interfaces, which ultimately enables more reliable LLM applications in healthcare.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。