提升医学领域少样本学习效果,通过多样性选例优化提示质量。
Data-Efficient Biomedical In-Context Learning: A Diversity-Enhanced Submodular Perspective
- 两阶段筛选:先兼顾代表性与多样性找候选例,再按相关性排序
- 在3项医学任务上最高提升5%的宏F1值,3-5个示例效率最优
- 对提示顺序和类别不平衡鲁棒,适合医疗文本少样本场景
大语言模型(LLM)的上下文学习(ICL)能力使其能快速适应未见的生物医学自然语言处理任务。仅需将少量输入-输出示例加入提示,即可实现新任务的快速执行。尽管已有研究关注示范例对模型性能的影响,但多数方法在从大规模语料中选取示例时更注重代表性而忽视多样性。为此,本文提出Dual-Div框架,一种增强多样性的数据高效示范例选择方法。该方法采用两阶段检索与排序机制:首先,在优化代表性和多样性基础上(可选对无标注数据进行标注),从语料库中识别有限候选示例;其次,针对测试查询对候选示例进行排序,选出最相关且非冗余的示范。在命名实体识别(NER)、关系抽取(RE)和文本分类(TC)三项生物医学任务上,使用LLaMA 3.1与Qwen 2.5进行推理,并结合BGE-Large、BMRetriever、MedCPT三种检索器评估,Dual-Div始终优于基线,最高实现5%的宏F1提升,且对提示顺序变化和类别不平衡具有鲁棒性。结果表明,初始检索阶段的多样性比排序阶段优化更为关键,将示范例限制在3-5个时性能效率最佳。
原文摘要 · Abstract (English)
Recent progress in large language models (LLMs) has leveraged their in-context learning (ICL) abilities to enable quick adaptation to unseen biomedical NLP tasks. By incorporating only a few input-output examples into prompts, LLMs can rapidly perform these new tasks. While the impact of these demonstrations on LLM performance has been extensively studied, most existing approaches prioritize representativeness over diversity when selecting examples from large corpora. To address this gap, we propose Dual-Div, a diversity-enhanced data-efficient framework for demonstration selection in biomedical ICL. Dual-Div employs a two-stage retrieval and ranking process: First, it identifies a limited set of candidate examples from a corpus by optimizing both representativeness and diversity (with optional annotation for unlabeled data). Second, it ranks these candidates against test queries to select the most relevant and non-redundant demonstrations. Evaluated on three biomedical NLP tasks (named entity recognition (NER), relation extraction (RE), and text classification (TC)) using LLaMA 3.1 and Qwen 2.5 for inference, along with three retrievers (BGE-Large, BMRetriever, MedCPT), Dual-Div consistently outperforms baselines-achieving up to 5% higher macro-F1 scores-while demonstrating robustness to prompt permutations and class imbalance. Our findings establish that diversity in initial retrieval is more critical than ranking-stage optimization, and limiting demonstrations to 3-5 examples maximizes performance efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。