让实体识别支持百万级标签，效率提升130倍。
The Million-Label NER: Breaking Scale Barriers with GLiNER bi-encoder
- 用双编码器分离标签与上下文编码，突破传统方法的计算瓶颈。
- 零样本性能达CrossNER基准61.5% Micro-F1，支持万级标签实时处理。
- 适合需要大规模实体识别与链接的工业级应用，如知识图谱构建。
本文提出GLiNER-bi-Encoder，一种新型命名实体识别架构，兼顾零样本泛化能力与工业级效率。原有GLiNER采用联合编码，标签数增加时复杂度呈平方增长。新设计将过程解耦为专用标签编码器与上下文编码器，有效消除上下文窗口瓶颈。该架构可同时识别数千甚至数百万种实体类型，开销极低。实验表明，其在CrossNER基准上达到61.5%的Micro-F1零样本性能。通过预计算标签嵌入，相比单编码器版本，在1024个标签下吞吐量提升高达130倍。此外，我们还引入GLiNKER框架，基于该架构实现对Wikidata等超大规模知识库的高性能实体链接。
原文摘要 · Abstract (English)
This paper introduces GLiNER-bi-Encoder, a novel architecture for Named Entity Recognition (NER) that harmonizes zero-shot flexibility with industrial-scale efficiency. While the original GLiNER framework offers strong generalization, its joint-encoding approach suffers from quadratic complexity as the number of entity labels increases. Our proposed bi-encoder design decouples the process into a dedicated label encoder and a context encoder, effectively removing the context-window bottleneck. This architecture enables the simultaneous recognition of thousands, and potentially millions, of entity types with minimal overhead. Experimental results demonstrate state-of-the-art zero-shot performance, achieving 61.5 percent Micro-F1 on the CrossNER benchmark. Crucially, by leveraging pre-computed label embeddings, GLiNER-bi-Encoder achieves up to a 130 times throughput improvement at 1024 labels compared to its uni-encoder predecessors. Furthermore, we introduce GLiNKER, a modular framework that leverages this architecture for high-performance entity linking across massive knowledge bases such as Wikidata.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。