首个面向大模型的动态细粒度命名实体识别数据集,支持多语言多层级标注。
DynamicNER: A Dynamic, Multilingual, and Fine-Grained Dataset for LLM-based Named Entity Recognition
- 动态实体分类:同一实体在不同上下文中可对应多种类型
- 覆盖8种语言155类实体,涵盖多元领域文本
- 适合研究大模型泛化能力与低资源场景下的命名实体识别
大语言模型(LLM)的发展推动了其在命名实体识别(NER)中的应用,但现有数据集主要针对传统机器学习方法设计,难以评估大模型的泛化与上下文理解能力。为解决这一问题,我们提出DynamicNER,首个专为大模型设计的动态、多语言、细粒度NER数据集,支持同一实体在不同上下文中具有多种类别,并覆盖8种语言和155种实体类型,涵盖多样化领域。同时,我们提出CascadeNER,一种基于两阶段策略与轻量级大模型的新型NER方法,在细粒度任务上实现更高精度且计算开销更低。实验表明,DynamicNER能有效评估大模型在不同条件下的表现。代码与数据集已开源。
原文摘要 · Abstract (English)
The advancements of Large Language Models (LLMs) have spurred a growing interest in their application to Named Entity Recognition (NER) methods. However, existing datasets are primarily designed for traditional machine learning methods and are inadequate for LLM-based methods, in terms of corpus selection and overall dataset design logic. Moreover, the prevalent fixed and relatively coarse-grained entity categorization in existing datasets fails to adequately assess the superior generalization and contextual understanding capabilities of LLM-based methods, thereby hindering a comprehensive demonstration of their broad application prospects. To address these limitations, we propose DynamicNER, the first NER dataset designed for LLM-based methods with dynamic categorization, introducing various entity types and entity type lists for the same entity in different context, leveraging the generalization of LLM-based NER better. The dataset is also multilingual and multi-granular, covering 8 languages and 155 entity types, with corpora spanning a diverse range of domains. Furthermore, we introduce CascadeNER, a novel NER method based on a two-stage strategy and lightweight LLMs, achieving higher accuracy on fine-grained tasks while requiring fewer computational resources. Experiments show that DynamicNER serves as a robust and effective benchmark for LLM-based NER methods. Furthermore, we also conduct analysis for traditional methods and LLM-based methods on our dataset. Our code and dataset are openly available at https://github.com/Astarojth/DynamicNER.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。