让大模型两次遍历文本，用双向上下文提升零样本命名实体识别准确率。
Just Pass Twice: Efficient Token Classification with LLMs for Zero-Shot NER
- 输入重复两次，让每个词在第二遍中看到完整上下文。
- 在CrossNER和MIT基准上平均F1提升7.9，超越此前最优方法。
- 速度超生成式方法20倍以上，适合实时应用与低资源场景。
大语言模型蕴含丰富的世界知识，对零样本命名实体识别（NER）很有价值。但其因果注意力机制限制了对后续上下文的利用，难以进行有效消歧。现有方法依赖生成式提示，存在解码慢、幻觉实体和格式错误等问题。本文提出只需两次遍历（Just Pass Twice, JPT）的方法，通过将输入重复拼接，使每个词在第二遍中获得完整的双向上下文，无需修改模型结构。结合定义引导的实体嵌入，实现灵活的零样本泛化。在跨领域和MIT基准上，平均F1比之前最佳方法高出7.9，且速度超过生成式方法20倍以上。交互式演示与模型可访问：https://witness-ai-jpt-ner.hf.space/
原文摘要 · Abstract (English)
Large language models encode extensive world knowledge valuable for zero-shot named entity recognition. However, their causal attention mechanism, where tokens attend only to preceding context, prevents effective token classification when disambiguation requires future context. Existing approaches use LLMs generatively, prompting them to list entities or produce structured outputs, but suffer from slow autoregressive decoding, hallucinated entities, and formatting errors. We propose Just Pass Twice (JPT), a simple yet effective method that enables causal LLMs to perform discriminative token classification with full bidirectional context. Our key insight is that concatenating the input to itself lets each token in the second pass attend to the complete sentence, requiring no architectural modifications. We combine these representations with definition-guided entity embeddings for flexible zero-shot generalization. Our approach achieves state-of-the-art results on zero-shot NER benchmarks, surpassing the previous best method by +7.9 F1 on average across CrossNER and MIT benchmarks, being over 20x faster than comparable generative methods. Interactive demo and access to the model are available at https://witness-ai-jpt-ner.hf.space/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。