构建首个NLP领域全文实体关系抽取基准,助力科学知识图谱构建。
SciNLP: A Domain-Specific Benchmark for Full-Text Scientific Entity and Relation Extraction in NLP
- 基于60篇完整论文人工标注,覆盖6409个实体与1648个关系。
- 模型在不同长度学术文本上表现差异显著,验证数据集挑战性。
- 支持构建细粒度知识图谱,平均节点度数达3.3,适合研究者使用。
从科学文献中结构化提取信息对捕捉专业领域核心概念与新兴趋势至关重要。现有数据集多聚焦于特定文章部分,受限于领域复杂性和标注成本。为此,我们提出SciNLP——首个面向自然语言处理(NLP)领域的全文实体与关系抽取基准。该数据集包含60篇经人工标注的完整NLP论文,涵盖6,409个实体和1,648个关系。相比现有研究,SciNLP是首个提供NLP领域全文标注的资源。我们通过对比实验验证其有效性,评估了先进监督模型在该数据集上的表现,结果表明现有模型在不同长度学术文本中的抽取能力存在差异。与已有数据集交叉比较显示,某些基线模型在SciNLP上取得显著提升。利用在SciNLP上训练的模型,我们实现了对NLP领域细粒度知识图谱的自动构建,该知识图谱平均每个实体关联3.3个其他实体,体现出丰富的语义拓扑结构,可增强下游应用效果。数据集已公开:https://github.com/AKADDC/SciNLP。
原文摘要 · Abstract (English)
Structured information extraction from scientific literature is crucial for capturing core concepts and emerging trends in specialized fields. While existing datasets aid model development, most focus on specific publication sections due to domain complexity and the high cost of annotating scientific texts. To address this limitation, we introduce SciNLP - a specialized benchmark for full-text entity and relation extraction in the Natural Language Processing (NLP) domain. The dataset comprises 60 manually annotated full-text NLP publications, covering 6,409 entities and 1,648 relations. Compared to existing research, SciNLP is the first dataset providing full-text annotations of entities and their relationships in the NLP domain. To validate the effectiveness of SciNLP, we conducted comparative experiments with similar datasets and evaluated the performance of state-of-the-art supervised models on this dataset. Results reveal varying extraction capabilities of existing models across academic texts of different lengths. Cross-comparisons with existing datasets show that SciNLP achieves significant performance improvements on certain baseline models. Using models trained on SciNLP, we implemented automatic construction of a fine-grained knowledge graph for the NLP domain. Our KG has an average node degree of 3.3 per entity, indicating rich semantic topological information that enhances downstream applications. The dataset is publicly available at: https://github.com/AKADDC/SciNLP.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。