对比BERT和T5在命名实体识别中的表现，发现T5在少样本下更优。
From BERT to T5: A Study of Named Entity Recognition
- 用BERT做编码器+分类头，T5做序列到序列生成+提示学习
- T5在3类简化标签上比BERT高2.1个点，少样本效果更好
- 适合研究模型在序列标注任务中的迁移能力与优化策略
命名实体识别（NER）是现代自然语言处理应用中的关键预处理步骤。本报告针对两个预训练模型进行微调实现NER任务：(i) 编码器仅有的模型BERT，搭配简单的分类头；(ii) 序列到序列模型T5，结合少样本提示方法。在原始7类标签与3类简化标签两种设定下，BERT采用加权交叉熵作为训练损失，T5则使用两种验证策略进行微调。同时进行了不同超参数的消融实验。相关分析揭示了BERT常见错误模式及两模型性能差异。基于多种性能指标，本报告旨在比较上述两种架构在序列标注任务中的表现，为后续实际应用场景提供基础支持。
原文摘要 · Abstract (English)
Named entity recognition (NER) has been one of the essential preliminary steps in modern NLP applications. This report focuses on implementing the NER task on finetuning two pretrained models: (i) an encoder-only model (BERT) with a simple classification head, and (ii) a sequence-to-sequence model (T5) with few-shot prompts. Under the original 7-class tag and 3-class simplified tag schemes, BERT is applied a weighted cross-entropy for training loss, and T5 is fine-tuned with two validation strategies. It also conducted an ablation study with different hyperparameters. Moreover, the related analysis provides valuable insights into common errors in BERT and the two models' performance. Based on a bunch of performance metrics, this report aims to compare the above two architectures and explore their abilities in the sequence labelling task, laying the groundwork for further practical use cases.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。