用ChatGPT做裁判,融合五种模型提升医疗文本断续实体识别效果。
On Fusing ChatGPT and Ensemble Learning in Discon-tinuous Named Entity Recognition in Health Corpora
- 设计基于提示工程的ChatGPT仲裁机制,融合五种先进命名实体识别模型。
- 在CADEC、ShARe13和ShARe14数据集上超越现有最优模型表现。
- 适合关注医疗NLP、模型融合与大模型应用的研究者参考。
命名实体识别是自然语言处理中的核心任务,旨在从非结构化文本中识别并提取重要术语。然而,当前深度学习NER模型在识别断续实体(即文本中分散的实体)方面仍面临显著挑战。据我们所知,尚未有研究探索将集成学习方法用于解决断续命名实体识别(DNER)问题。近年来,大型语言模型如ChatGPT在多个NLP任务中表现出色,但大多数方法仅将其作为问题求解工具,未深入挖掘其作为集成学习中整合组件的潜力。本研究提出将ChatGPT作为仲裁器,融入集成学习框架,以增强DNER任务性能。方法结合五个最先进的NER模型,并通过定制提示工程让ChatGPT评估各模型输出,从而实现更鲁棒的集成决策。我们在三个基准医疗数据集(CADEC、ShARe13、ShARe14)上进行实验,对比了所提方法与五种SOTA模型、GPT-3.5和GPT-4独立使用结果,以及投票集成方法。结果表明,该融合方法在上述三个数据集上的表现均优于现有最优结果,展现出在医疗领域NLP应用中的巨大潜力。
原文摘要 · Abstract (English)
Named Entity Recognition has traditionally been a key task in natural language processing, aiming to identify and extract important terms from unstructured text data. However, a notable challenge for contemporary deep-learning NER models has been identifying discontinuous entities, which are often fragmented within the text. To date, methods to address Discontinuous Named Entity Recognition have not been explored using ensemble learning to the best of our knowledge. Furthermore, the rise of large language models, such as ChatGPT in recent years, has shown significant effectiveness across many NLP tasks. Most existing approaches, however, have primarily utilized ChatGPT as a problem-solving tool rather than exploring its potential as an integrative element within ensemble learning algorithms. In this study, we investigated the integration of ChatGPT as an arbitrator within an ensemble method, aiming to enhance performance on DNER tasks. Our method combines five state-of-the-art NER models with ChatGPT using custom prompt engineering to assess the robustness and generalization capabilities of the ensemble algorithm. We conducted experiments on three benchmark medical datasets, comparing our method against the five SOTA models, individual applications of GPT-3.5 and GPT-4, and a voting ensemble method. The results indicate that our proposed fusion of ChatGPT with the ensemble learning algorithm outperforms the SOTA results in the CADEC, ShARe13, and ShARe14 datasets, showcasing its potential to enhance NLP applications in the healthcare domain.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。