大模型让命名实体识别更智能,无需复杂结构也能高效准确。
Assessment of Generative Named Entity Recognition in the Era of Large Language Models
- 用指令微调+结构化输出,大模型在实体识别上表现媲美传统方法。
- 大模型靠理解指令而非记忆数据,泛化能力更强。
- 专用于实体识别的训练几乎不影响模型其他能力,反而提升部分任务性能。
随着大语言模型(LLMs)的发展,命名实体识别(NER)正从序列标注转向生成式范式。本文系统评估了八种不同规模的开源大模型在平坦和嵌套NER任务上的表现。通过四个标准数据集实验发现:(1)采用参数高效微调与括号或XML等结构化输出格式,开源大模型性能可媲美传统编码器模型,并超越仅依赖上下文学习的解码器模型;(2)大模型的NER能力源于指令遵循与生成能力,而非对实体标签对的单纯记忆;(3)进行NER指令微调对模型通用能力影响极小,甚至在DROP数据集上使F1分数提升25.50至45.32点,因实体理解能力增强。结果表明,基于大模型的生成式NER是一种有前景且用户友好的替代方案。相关数据与代码已开源于https://github.com/szu-tera/LLMs4NER。
原文摘要 · Abstract (English)
Named entity recognition (NER) is evolving from a sequence labeling task into a generative paradigm with the rise of large language models (LLMs). We conduct a systematic evaluation of open-source LLMs on both flat and nested NER tasks. We investigate several research questions including the performance gap between generative NER and traditional NER models, the impact of output formats, whether LLMs rely on memorization, and the preservation of general capabilities after fine-tuning. Through experiments across eight LLMs of varying scales and four standard NER datasets, we find that: (1) With parameter-efficient fine-tuning and structured formats like inline bracketed or XML, open-source LLMs achieve performance competitive with traditional encoder-based models and surpass decoder-based LLMs with in-context learning techniques; (2) The NER capability of LLMs stems from instruction-following and generative power, not mere memorization of entity-label pairs; and (3) Applying NER instruction tuning has minimal impact on general capabilities of LLMs, even improving performance on datasets like DROP by 25.50 to 45.32 F1 points due to enhanced entity understanding. These findings demonstrate that generative NER with LLMs is a promising, user-friendly alternative to traditional methods. We release the data and code at https://github.com/szu-tera/LLMs4NER.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。