用不到300个参数的轻量模型，从大模型早期层中高效提取实体提及信息。
ToMMeR -- Efficient Entity Mention Detection from Large Language Models
- 通过探测大模型早期层，仅用300K参数实现高精度实体提及识别。
- 零样本下召回率达93%，人工校准后精确率预估达90%。
- 适合研究大模型内部表示或需轻量级实体识别的场景。
识别文本中指向实体的词段——提及检测——是信息抽取的基础，也是性能瓶颈。我们提出ToMMeR，一个参数少于300K的轻量模型，用于探测大语言模型早期层中的提及检测能力。在13个NER基准上，ToMMeR实现了93%的零样本召回率，在人工校准的大模型评判协议下估计精确率为90%，表明其虽召回率高但极少产生误报。跨模型分析显示，不同架构（14M-15B参数）均收敛到相似的提及边界（DICE >75%），证实提及检测自然涌现于语言建模过程。当添加跨度分类头后，ToMMeR在标准基准上达到80-87%的F1值，表现具有竞争力。本工作表明，结构化实体表示存在于早期Transformer层中，且可用极小参数高效恢复。
原文摘要 · Abstract (English)
Identifying which text spans refer to entities - mention detection - is both foundational for information extraction and a known performance bottleneck. We introduce ToMMeR, a lightweight model (<300K parameters) probing mention detection capabilities from early LLM layers. Across 13 NER benchmarks, ToMMeR achieves 93% recall zero-shot, with an estimated 90% precision under a human-calibrated LLM-judge protocol, showing that ToMMeR rarely produces spurious predictions despite high recall. Cross-model analysis reveals that diverse architectures (14M-15B parameters) converge on similar mention boundaries (DICE >75%), confirming that mention detection emerges naturally from language modeling. When extended with span classification heads, ToMMeR achieves competitive NER performance (80-87% F1 on standard benchmarks). Our work provides evidence that structured entity representations exist in early transformer layers and can be efficiently recovered with minimal parameters.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。