用检索增强生成技术提升大模型对药物禁忌的判断准确率
Retrieval Augmented Large Language Model System for Comprehensive Drug Contraindications
- 引入RAG框架，结合公开数据库与重排序机制检索药物禁忌信息
- 对年龄、孕期及联合用药禁忌的准确率分别提升至94%、87%和89%
- 适合医疗AI研发者或临床决策支持系统开发者参考
大语言模型在医疗领域的应用面临挑战，尤其在药物禁忌信息的准确性要求极高。本研究通过构建检索增强生成（RAG）系统，提升LLM在药物禁忌判断中的表现。以OpenAI的GPT-4o-mini为基底模型，使用text-embedding-3-small进行文本嵌入，利用Langchain实现混合检索与重排序机制。系统整合了来自公共数据库的药物利用审查（DUR）数据，聚焦特定年龄组、妊娠期及合并用药的禁忌信息。数据集包含300个问答对，覆盖三个类别，原始模型准确率在0.49至0.57之间。集成RAG后，针对年龄组、妊娠期和合并用药禁忌的准确率分别达到0.94、0.87和0.89。结果表明，RAG框架能显著降低处方与用药决策中的不确定性，提供更精准可靠的药物禁忌信息。
原文摘要 · Abstract (English)
The versatility of large language models (LLMs) has been explored across various sectors, but their application in healthcare poses challenges, particularly in the domain of pharmaceutical contraindications where accurate and reliable information is required. This study enhances the capability of LLMs to address contraindications effectively by implementing a Retrieval Augmented Generation (RAG) pipeline. Utilizing OpenAI's GPT-4o-mini as the base model, and the text-embedding-3-small model for embeddings, our approach integrates Langchain to orchestrate a hybrid retrieval system with re-ranking. This system leverages Drug Utilization Review (DUR) data from public databases, focusing on contraindications for specific age groups, pregnancy, and concomitant drug use. The dataset includes 300 question-answer pairs across three categories, with baseline model accuracy ranging from 0.49 to 0.57. Post-integration of the RAG pipeline, we observed a significant improvement in model accuracy, achieving rates of 0.94, 0.87, and 0.89 for contraindications related to age groups, pregnancy, and concomitant drug use, respectively. The results indicate that augmenting LLMs with a RAG framework can substantially reduce uncertainty in prescription and drug intake decisions by providing more precise and reliable drug contraindication information.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。