用生成式框架解决持续学习中的实体识别遗忘问题
GenCNER: A Generative Framework for Continual Named Entity Recognition
- 将连续实体识别转为实体三元组生成任务，利用预训练序列模型求解
- 通过置信度伪标签和知识蒸馏，减少非实体类别的语义偏移与噪声影响
- 在两个基准数据集上表现超越现有方法，接近非持续学习的性能
传统命名实体识别（NER）旨在将文本提及分类到预定义实体类型中。由于现实场景中实体类别持续增加，连续命名实体识别（CNER）应运而生。然而，现有面向NER的持续学习（CL）方法面临灾难性遗忘和非实体类别的语义漂移问题。本文提出GenCNER，一种简单但高效的生成式框架，用于缓解上述挑战。具体而言，我们巧妙地将CNER任务转化为持续的实体三元组序列生成问题，并利用强大的预训练序列到序列模型求解。此外，我们设计了一种基于类型置信度的伪标签策略，结合知识蒸馏（KD），以保留已学知识并减轻三元组级别标签噪声的影响。在两个基准数据集上的实验结果表明，我们的框架在多种CNER设置下优于先前最优方法，且与非持续学习结果的差距最小。
原文摘要 · Abstract (English)
Traditional named entity recognition (NER) aims to identify text mentions into pre-defined entity types. Continual Named Entity Recognition (CNER) is introduced since entity categories are continuously increasing in various real-world scenarios. However, existing continual learning (CL) methods for NER face challenges of catastrophic forgetting and semantic shift of non-entity type. In this paper, we propose GenCNER, a simple but effective Generative framework for CNER to mitigate the above drawbacks. Specifically, we skillfully convert the CNER task into sustained entity triplet sequence generation problem and utilize a powerful pre-trained seq2seq model to solve it. Additionally, we design a type-specific confidence-based pseudo labeling strategy along with knowledge distillation (KD) to preserve learned knowledge and alleviate the impact of label noise at the triplet level. Experimental results on two benchmark datasets show that our framework outperforms previous state-of-the-art methods in multiple CNER settings, and achieves the smallest gap compared with non-CL results.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。