用预训练模型+少样本学习提升医学实体识别准确率。
Pre-trained Language Models and Few-shot Learning for Medical Entity Extraction
- 基于Transformer架构，对比多种医学预训练模型效果。
- PubMedBERT达88.8%的F1分数，Span方法最佳（88.6%）。
- 仅需10个样本即达79.1%准确率，适合数据稀缺场景。
本研究提出一种基于Transformer的医学实体提取方法，以提升医学文献的信息抽取能力。针对医学文本的专业性与复杂性，对比了BERT、BioBERT、PubMedBERT和ClinicalBERT等预训练语言模型在医学实体抽取任务中的表现。实验结果显示，PubMedBERT取得最佳性能（F1-score = 88.8%），表明在生物医学文献上预训练的语言模型在医疗领域更具优势。同时，分析不同实体抽取方法（CRF、Span-based、Seq2Seq）发现，Span-based方法表现最优（F1-score = 88.6%），在识别实体边界方面具有更高精度。在低资源场景下，进一步探索了少样本学习的应用效果。结果表明，即使仅使用10个训练样本，模型仍可达到79.1%的F1分数，验证了少样本学习在数据有限条件下的有效性。研究证实，预训练模型与少样本学习结合能显著提升医学实体抽取的准确性。未来可融合知识图谱与主动学习策略，增强模型泛化与稳定性，为医学自然语言处理提供更高效解决方案。
原文摘要 · Abstract (English)
This study proposes a medical entity extraction method based on Transformer to enhance the information extraction capability of medical literature. Considering the professionalism and complexity of medical texts, we compare the performance of different pre-trained language models (BERT, BioBERT, PubMedBERT, ClinicalBERT) in medical entity extraction tasks. Experimental results show that PubMedBERT achieves the best performance (F1-score = 88.8%), indicating that a language model pre-trained on biomedical literature is more effective in the medical domain. In addition, we analyze the impact of different entity extraction methods (CRF, Span-based, Seq2Seq) and find that the Span-based approach performs best in medical entity extraction tasks (F1-score = 88.6%). It demonstrates superior accuracy in identifying entity boundaries. In low-resource scenarios, we further explore the application of Few-shot Learning in medical entity extraction. Experimental results show that even with only 10-shot training samples, the model achieves an F1-score of 79.1%, verifying the effectiveness of Few-shot Learning under limited data conditions. This study confirms that the combination of pre-trained language models and Few-shot Learning can enhance the accuracy of medical entity extraction. Future research can integrate knowledge graphs and active learning strategies to improve the model's generalization and stability, providing a more effective solution for medical NLP research. Keywords- Natural Language Processing, medical named entity recognition, pre-trained language model, Few-shot Learning, information extraction, deep learning
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。