用大模型做历史文本命名实体识别,效果更好且无需编程
NER4all or Context is All You Need: Using LLMs for low-effort, high-performance NER on historical texts. A humanities informed approach
- 用大模型+历史背景提示,替代传统NLP工具
- F1分数比spaCy和flair高7%到22%
- 16次示例后增加样本不再提升效果,适合非技术研究者
命名实体识别(NER)是历史研究中自动识别人物、地点、事件等关键信息的核心任务。然而,由于史料语言多样、拼写不统一、需大量历史知识且标注数据稀缺,传统自然语言处理方法成本高昂,召回率与精确率均不理想。本文提出新方法:利用现成的先进大模型,在历史文本上实现显著优于spaCy和flair的NER性能,F1得分提升7%至22%。消融实验表明,提供历史上下文并引入少量人物角色建模,是有效提示策略的关键。此外,我们发现少样本学习中,示例数量超过16个后,召回率与精确率不再提升。该方法降低了使用门槛,使所有历史学者无需编程或计算技能,仅通过自然语言提示即可高效完成NER任务。
原文摘要 · Abstract (English)
Named entity recognition (NER) is a core task for historical research in automatically establishing all references to people, places, events and the like. Yet, do to the high linguistic and genre diversity of sources, only limited canonisation of spellings, the level of required historical domain knowledge, and the scarcity of annotated training data, established approaches to natural language processing (NLP) have been both extremely expensive and yielded only unsatisfactory results in terms of recall and precision. Our paper introduces a new approach. We demonstrate how readily-available, state-of-the-art LLMs significantly outperform two leading NLP frameworks, spaCy and flair, for NER in historical documents by seven to twentytwo percent higher F1-Scores. Our ablation study shows how providing historical context to the task and a bit of persona modelling that turns focus away from a purely linguistic approach are core to a successful prompting strategy. We also demonstrate that, contrary to our expectations, providing increasing numbers of examples in few-shot approaches does not improve recall or precision below a threshold of 16-shot. In consequence, our approach democratises access to NER for all historians by removing the barrier of scripting languages and computational skills required for established NLP tools and instead leveraging natural language prompts and consumer-grade tools and frontends.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。