用大模型提升罕见病实体识别,少样本下效果媲美专业模型。
Leveraging Large Language Models for Rare Disease Named Entity Recognition
- 设计结构化提示框架,融合领域知识与消歧规则。
- 微调版GPT-4o在罕见病语料上超越现有最佳模型(BioClinicalBERT)。
- 少样本提示性价比高,适合数据稀缺场景。
罕见病领域的命名实体识别因标注数据稀少、实体类型语义模糊及长尾分布而面临挑战。本研究评估GPT-4o在低资源条件下对罕见病NER的表现,采用零样本提示、少样本上下文学习、检索增强生成(RAG)及任务级微调等多种策略。设计了包含四种实体类型领域知识与消歧规则的结构化提示框架,并提出两种语义引导的少样本示例选择方法,以降低标注成本。在RareDis Corpus上的实验表明,GPT-4o表现优于或媲美BioClinicalBERT,其中任务级微调效果最佳,超过此前报道的BioClinicalBERT基线。成本-性能分析显示,少样本提示在低令牌预算下收益显著。RAG虽整体增益有限,但可提升难识别实体(如症状)的召回率。错误分类分析揭示边界漂移和类型混淆是主要失败模式,提示可通过后处理与混合优化改进。结果表明,经提示优化的大模型可成为生物医学命名实体识别中传统监督模型的有效替代方案,尤其适用于标注数据稀缺的罕见病场景。
原文摘要 · Abstract (English)
Named Entity Recognition (NER) in the rare disease domain poses unique challenges due to limited labeled data, semantic ambiguity between entity types, and long-tail distributions. In this study, we evaluate the capabilities of GPT-4o for rare disease NER under low-resource settings, using a range of prompt-based strategies including zero-shot prompting, few-shot in-context learning, retrieval-augmented generation (RAG), and task-level fine-tuning. We design a structured prompting framework that encodes domain-specific knowledge and disambiguation rules for four entity types. We further introduce two semantically guided few-shot example selection methods to improve in-context performance while reducing labeling effort. Experiments on the RareDis Corpus show that GPT-4o achieves competitive or superior performance compared to BioClinicalBERT, with task-level fine-tuning yielding the strongest performance among the evaluated approaches and improving upon the previously reported BioClinicalBERT baseline. Cost-performance analysis reveals that few-shot prompting delivers high returns at low token budgets. RAG provides limited overall gains but can improve recall for challenging entity types, especially signs and symptoms. An error taxonomy highlights common failure modes such as boundary drift and type confusion, suggesting opportunities for post-processing and hybrid refinement. Our results demonstrate that prompt-optimized LLMs can serve as effective, scalable alternatives to traditional supervised models in biomedical NER, particularly in rare disease applications where annotated data is scarce.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。