用检索增强动态提示，提升小样本生物医学命名实体识别效果
Retrieval augmented generation based dynamic prompting for few-shot biomedical named entity recognition using large language models
- 根据输入文本相似性动态选取上下文示例，生成自适应提示
- 5-10轮提示下，平均F1提升7.3%~11%，最大增益达12%
- 适合资源有限的生物医学信息提取场景，尤其适配大模型
生物医学命名实体识别（NER）是高价值自然语言处理任务，大语言模型（LLMs）在小样本设置下表现突出。本文针对LLMs在小样本生物医学NER中的性能挑战，提出基于检索增强生成（RAG）的动态提示策略。方法通过计算输入文本与标注示例的相似度，动态选择上下文学习样例，并在推理时为每个实例更新提示。对比静态与动态提示技术，结构化静态提示使GPT-4、GPT-3.5和LLaMA 3-70B的平均F1分别提升12%、11%；动态提示进一步优化，在5-shot和10-shot设置下分别实现7.3%和5.6%的平均F1提升。结果表明，基于RAG的上下文自适应提示显著增强模型性能。
原文摘要 · Abstract (English)
Biomedical named entity recognition (NER) is a high-utility natural language processing (NLP) task, and large language models (LLMs) show promise particularly in few-shot settings (i.e., limited training data). In this article, we address the performance challenges of LLMs for few-shot biomedical NER by investigating a dynamic prompting strategy involving retrieval-augmented generation (RAG). In our approach, the annotated in-context learning examples are selected based on their similarities with the input texts, and the prompt is dynamically updated for each instance during inference. We implemented and optimized static and dynamic prompt engineering techniques and evaluated them on five biomedical NER datasets. Static prompting with structured components increased average F1-scores by 12% for GPT-4, and 11% for GPT-3.5 and LLaMA 3-70B, relative to basic static prompting. Dynamic prompting further improved performance, with TF-IDF and SBERT retrieval methods yielding the best results, improving average F1-scores by 7.3% and 5.6% in 5-shot and 10-shot settings, respectively. These findings highlight the utility of contextually adaptive prompts via RAG for biomedical NER.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。