arXiv:2412.19346cs.CLcs.AI2024-12

用少量标注数据+大量无标注数据,实现临床文献中细粒度PICO实体精准识别。

Semi-Supervised Learning from Small Annotated Data and Large Unlabeled Data for Fine-grained PICO Entity Recognition

  • 结合少量标注与大量无标注数据,设计半监督训练方法
  • 仅用小样本标注数据即达F1 0.60,比基线高16%以上
  • 适用于不同临床框架,适合医学信息提取研究者

目的:从临床试验文献中提取参与者(Participants)、干预措施(Intervention)、对照组(Comparison)和结局(Outcomes)——即PICO元素——对临床证据检索、评估与整合至关重要。现有方法未区分PICO实体的属性。本研究旨在开发一种命名实体识别(NER)模型,以细粒度提取PICO实体。方法:基于4个公开数据集共2,511篇摘要中带有PICO标注的语料,我们提出一种半监督方法,通过结合有限标注数据与大量无标注数据来训练名为FinePICO的NER模型。评估时,将数据集分为小标注子集与大无标注子集,并分别建立仅使用小标注集与全标注集训练的监督学习模型的理论性能下界与上界。最终在小标注集与大无标注集上评估FinePICO表现,采用精确率、召回率与F1值衡量。结果:该方法在小样本标注数据下取得0.567/0.636/0.60的精确率/召回率/F1,较基线模型(F1: 0.437)提升超16%;且在不同PICO框架与另一语料上均具泛化能力,实验结果差异显著(p-value < 0.001)。结论:本研究提出一种可泛化且高效的半监督命名实体识别方法,能有效利用大量无标注数据与少量标注数据,初步支持细粒度PICO提取。

原文摘要 · Abstract (English)

Objective: Extracting PICO elements -- Participants, Intervention, Comparison, and Outcomes -- from clinical trial literature is essential for clinical evidence retrieval, appraisal, and synthesis. Existing approaches do not distinguish the attributes of PICO entities. This study aims to develop a named entity recognition (NER) model to extract PICO entities with fine granularities. Materials and Methods: Using a corpus of 2,511 abstracts with PICO mentions from 4 public datasets, we developed a semi-supervised method to facilitate the training of a NER model, FinePICO, by combining limited annotated data of PICO entities and abundant unlabeled data. For evaluation, we divided the entire dataset into two subsets: a smaller group with annotations and a larger group without annotations. We then established the theoretical lower and upper performance bounds based on the performance of supervised learning models trained solely on the small, annotated subset and on the entire set with complete annotations, respectively. Finally, we evaluated FinePICO on both the smaller annotated subset and the larger, initially unannotated subset. We measured the performance of FinePICO using precision, recall, and F1. Results: Our method achieved precision/recall/F1 of 0.567/0.636/0.60, respectively, using a small set of annotated samples, outperforming the baseline model (F1: 0.437) by more than 16\%. The model demonstrates generalizability to a different PICO framework and to another corpus, which consistently outperforms the benchmark in diverse experimental settings (p-value \textless0.001). Conclusion: This study contributes a generalizable and effective semi-supervised approach to named entity recognition leveraging large unlabeled data together with small, annotated data. It also initially supports fine-grained PICO extraction.

PICO提取半监督学习医学NLP细粒度识别

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。