通过多模式检索优化医学文本生成中的示例选择。
MMRAG: Multi-Mode Retrieval-Augmented Generation with Large Language Models for Biomedical In-Context Learning
- 融合四种检索策略，智能挑选不同类型的示例。
- 在药物相互作用任务中F1达0.9669，提升26.4%。
- 适合医学NLP研究者与医疗AI开发者参考。
目标：优化生物医学自然语言处理中的上下文学习，通过改进示例选择。方法：提出一种多模式检索增强生成（MMRAG）框架，集成四种检索策略：随机模式（任意选取示例）、最优模式（基于相似性选最相关示例）、多样性模式（确保选取示例的差异性）、类别模式（选取类别代表性示例）。在三个核心生物医学NLP任务上评估：命名实体识别（NER）、关系抽取（RE）和文本分类（TC）。数据集包括用于基因蛋白识别的BC2GM、药物相互作用抽取的DDI、通用生物信息抽取的GIT，以及健康咨询文本分类的HealthAdvice。使用Llama2-7B和Llama3-8B两个大模型，搭配Contriever、MedCPT、BGE-Large三种检索器，测试不同策略表现。结果：随机模式显示，增加示例数量可提升生成性能；最优模式与多样性模式在关系抽取（DDI）任务上显著优于随机模式，F1达0.9669，提升26.4%。三种检索器中，Contriever在多数实验中表现更优。两模型表现各异，Llama3在命名实体识别任务中优势明显。结论：MMRAG通过优化示例选择，有效提升生物医学上下文学习效果，缓解数据稀缺问题，展现出对医疗AI应用的良好适应性。
原文摘要 · Abstract (English)
Objective: To optimize in-context learning in biomedical natural language processing by improving example selection. Methods: We introduce a novel multi-mode retrieval-augmented generation (MMRAG) framework, which integrates four retrieval strategies: (1) Random Mode, selecting examples arbitrarily; (2) Top Mode, retrieving the most relevant examples based on similarity; (3) Diversity Mode, ensuring variation in selected examples; and (4) Class Mode, selecting category-representative examples. This study evaluates MMRAG on three core biomedical NLP tasks: Named Entity Recognition (NER), Relation Extraction (RE), and Text Classification (TC). The datasets used include BC2GM for gene and protein mention recognition (NER), DDI for drug-drug interaction extraction (RE), GIT for general biomedical information extraction (RE), and HealthAdvice for health-related text classification (TC). The framework is tested with two large language models (Llama2-7B, Llama3-8B) and three retrievers (Contriever, MedCPT, BGE-Large) to assess performance across different retrieval strategies. Results: The results from the Random mode indicate that providing more examples in the prompt improves the model's generation performance. Meanwhile, Top mode and Diversity mode significantly outperform Random mode on the RE (DDI) task, achieving an F1 score of 0.9669, a 26.4% improvement. Among the three retrievers tested, Contriever outperformed the other two in a greater number of experiments. Additionally, Llama 2 and Llama 3 demonstrated varying capabilities across different tasks, with Llama 3 showing a clear advantage in handling NER tasks. Conclusion: MMRAG effectively enhances biomedical in-context learning by refining example selection, mitigating data scarcity issues, and demonstrating superior adaptability for NLP-driven healthcare applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。