小模型高效提取长文本实体，省显存还准
Small Language Model Makes an Effective Long Text Extractor
- 用轻量级方法结合双向注意力和滑动窗口，减少冗余计算
- 在三个长文本命名实体数据集上达到顶尖准确率
- 适合资源有限但需长跨度实体抽取的场景
命名实体识别（NER）是自然语言处理中的基础任务。然而，从长文本（如主页）中提取更长的实体跨度（如奖项）的研究仍较少。现有NER方法主要分为两类：基于跨度的方法需要枚举所有可能的标记对跨度并分类，导致大量冗余计算和高显存占用；基于生成的方法则依赖大模型提示或微调，难以准确生成长跨度且微调耗时长。本文提出一种轻量级基于跨度的NER方法SeNER，采用双向箭头注意力机制与[CLS] token的LogN-Scaling，有效嵌入长文本，并设计新型双向滑动窗口加号注意力（BiSPA）机制，显著减少冗余候选跨度，同时建模跨度间交互。大量实验表明，该方法在三个长文本NER数据集上达到当前最优准确率，且显存消耗友好。代码已开源。
原文摘要 · Abstract (English)
Named Entity Recognition (NER) is a fundamental problem in natural language processing (NLP). However, the task of extracting longer entity spans (e.g., awards) from extended texts (e.g., homepages) is barely explored. Current NER methods predominantly fall into two categories: span-based methods and generation-based methods. Span-based methods require the enumeration of all possible token-pair spans, followed by classification on each span, resulting in substantial redundant computations and excessive GPU memory usage. In contrast, generation-based methods involve prompting or fine-tuning large language models (LLMs) to adapt to downstream NER tasks. However, these methods struggle with the accurate generation of longer spans and often incur significant time costs for effective fine-tuning. To address these challenges, this paper introduces a lightweight span-based NER method called SeNER, which incorporates a bidirectional arrow attention mechanism coupled with LogN-Scaling on the [CLS] token to embed long texts effectively, and comprises a novel bidirectional sliding-window plus-shaped attention (BiSPA) mechanism to reduce redundant candidate token-pair spans significantly and model interactions between token-pair spans simultaneously. Extensive experiments demonstrate that our method achieves state-of-the-art extraction accuracy on three long NER datasets and is capable of extracting entities from long texts in a GPU-memory-friendly manner. Code: https://github.com/THUDM/scholar-profiling/tree/main/sener
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。