arXiv:2411.00451cs.LGcs.IR2024-11EMNLP被引 10

用指令微调增强检索的模型,让少样本跨领域实体识别更准更快。

Improving Few-Shot Cross-Domain Named Entity Recognition by Instruction Tuning a Word-Embedding based Retrieval Augmented Large Language Model

  • 基于词嵌入检索+指令微调,提升大模型在新领域的实体识别能力。
  • 在CrossNER数据集上比之前最优模型提升超2% F1值,跨域效果显著。
  • 部署后减少15%人工工单,企业年省数百万,适合实际业务场景落地。

少样本跨领域命名实体识别旨在利用数据丰富的源领域知识,在数据稀缺的目标领域进行实体识别。现有主流方法多依赖预训练语言模型(PLMs),但这些模型通常具有领域局限性,需针对新领域修改结构或微调,导致为每个目标领域生成全新模型,不适用于实际应用。近期虽有研究尝试使用大语言模型(LLM)解决该问题,但多数成本过高或难以遵循提示指令。本文提出IF-WRANER:一种基于词嵌入检索的指令微调型增强型大语言模型,用于命名实体识别。通过微调时的正则化技术及提示内检索采用词级嵌入而非句级嵌入,该模型在跨域任务中表现优异。我们在开源数据集CrossNER上验证其性能,相比前序SOTA模型,F1值提升超过2%。已在企业多个客户客服领域部署,准确预测实体使客户自动流转至对应流程,将人工转接率降低约15%,每年为企业节省数百万美元。

原文摘要 · Abstract (English)

Few-Shot Cross-Domain NER is the process of leveraging knowledge from data-rich source domains to perform entity recognition on data scarce target domains. Most previous state-of-the-art (SOTA) approaches use pre-trained language models (PLMs) for cross-domain NER. However, these models are often domain specific. To successfully use these models for new target domains, we need to modify either the model architecture or perform model finetuning using data from the new domains. Both of these result in the creation of entirely new NER models for each target domain which is infeasible for practical scenarios. Recently,several works have attempted to use LLMs to solve Few-Shot Cross-Domain NER. However, most of these are either too expensive for practical purposes or struggle to follow LLM prompt instructions. In this paper, we propose IF-WRANER (Instruction Finetuned Word-embedding based Retrieval Augmented large language model for Named Entity Recognition), a retrieval augmented LLM, finetuned for the NER task. By virtue of the regularization techniques used during LLM finetuning and the adoption of word-level embedding over sentence-level embedding during the retrieval of in-prompt examples, IF-WRANER is able to outperform previous SOTA Few-Shot Cross-Domain NER approaches. We have demonstrated the effectiveness of our model by benchmarking its performance on the open source CrossNER dataset, on which it shows more than 2% F1 score improvement over the previous SOTA model. We have deployed the model for multiple customer care domains of an enterprise. Accurate entity prediction through IF-WRANER helps direct customers to automated workflows for the domains, thereby reducing escalations to human agents by almost 15% and leading to millions of dollars in yearly savings for the company.

命名实体识别少样本学习大模型应用跨领域迁移

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。