首个法律文档事件共指消解数据集，挑战大模型理解长文本能力
LegalCore: A Dataset for Event Coreference Resolution in Legal Documents
- 构建法律合同事件共指标注数据集，单文档平均25k词元
- 法律文档存在密集事件提及与超远距离共指链接，模型表现显著下降
- 适合法律AI、长文本理解方向研究者使用
识别文本中事件及其共指提及对于理解语义至关重要。现有事件共指消解研究多集中于新闻文章。本文首次提出法律领域数据集LegalCore，对法律合同文档进行全面事件与共指标注。这些合同文档平均长度达约25,000词元，远超新闻文章。标注显示，法律文档具有密集事件提及，且存在短距离与超远距离共指关系。我们在该数据集上对主流大语言模型（LLMs）进行事件检测与共指消解任务评测，发现其表现显著低于监督基线，表明该数据集对当前先进开源与专有模型构成重大挑战。我们将公开数据集及代码。
原文摘要 · Abstract (English)
Recognizing events and their coreferential mentions in a document is essential for understanding semantic meanings of text. The existing research on event coreference resolution is mostly limited to news articles. In this paper, we present the first dataset for the legal domain, LegalCore, which has been annotated with comprehensive event and event coreference information. The legal contract documents we annotated in this dataset are several times longer than news articles, with an average length of around 25k tokens per document. The annotations show that legal documents have dense event mentions and feature both short-distance and super long-distance coreference links between event mentions. We further benchmark mainstream Large Language Models (LLMs) on this dataset for both event detection and event coreference resolution tasks, and find that this dataset poses significant challenges for state-of-the-art open-source and proprietary LLMs, which perform significantly worse than a supervised baseline. We will publish the dataset as well as the code.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。