评测大模型在历史文献长尾实体链接中的表现
Evaluation of LLMs on Long-tail Entity Linking in Historical Documents
- 用真实历史文本构建基准测试，对比大模型与传统方法
- 大模型在冷门实体链接任务中表现良好，超越传统框架
- 适合对历史文本挖掘、知识库补全感兴趣的学者
实体链接（EL）在自然语言处理中至关重要，可将文本中的实体提及消歧并链接至参考知识库条目。得益于深度上下文理解能力，大语言模型（LLMs）为解决EL问题提供了新思路，有望超越传统方法。尽管大模型泛化能力强，但在冷门、长尾实体链接任务中仍面临挑战，因这些实体在训练数据和知识库中代表性不足。此外，长尾实体链接研究较少，且鲜有工作使用大模型进行探索。本文评估了GPT与LLama3两款主流大模型在长尾实体链接任务中的表现，采用MHERCL v0.1这一手工标注的历史文本语料库，定量比较其在识别和链接到Wikidata条目方面的性能，相较于ReLiK框架。初步实验表明，大模型在长尾实体链接任务中表现令人鼓舞，证明该技术能有效弥补头部与长尾实体链接之间的差距。
原文摘要 · Abstract (English)
Entity Linking (EL) plays a crucial role in Natural Language Processing (NLP) applications, enabling the disambiguation of entity mentions by linking them to their corresponding entries in a reference knowledge base (KB). Thanks to their deep contextual understanding capabilities, LLMs offer a new perspective to tackle EL, promising better results than traditional methods. Despite the impressive generalization capabilities of LLMs, linking less popular, long-tail entities remains challenging as these entities are often underrepresented in training data and knowledge bases. Furthermore, the long-tail EL task is an understudied problem, and limited studies address it with LLMs. In the present work, we assess the performance of two popular LLMs, GPT and LLama3, in a long-tail entity linking scenario. Using MHERCL v0.1, a manually annotated benchmark of sentences from domain-specific historical texts, we quantitatively compare the performance of LLMs in identifying and linking entities to their corresponding Wikidata entries against that of ReLiK, a state-of-the-art Entity Linking and Relation Extraction framework. Our preliminary experiments reveal that LLMs perform encouragingly well in long-tail EL, indicating that this technology can be a valuable adjunct in filling the gap between head and long-tail EL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。