研究指出现有桥接指代标注存在主观偏差,影响一致性。
Subjectivity in the Annotation of Bridging Anaphora
- 在GUM语料测试集上开展标注实验,评估三类任务的主观性
- 发现桥接类型标注一致性中等,但实例识别重叠率低
- 多数分歧源于对实体理解的主观差异,旧资源可能严重漏标
桥接指代指话语中可推断实体与其先行词之间的关联关系,如理解‘门’与前文‘房子’的关系。由于该任务本质具有主观性,现有桥接指代的标注难以达成一致。本文在已有GUM语料测试集上开展标注试点,探讨了指代识别、先行词定位及桥接类型选择三个层面的主观性问题。我们提出一种新的桥接子类型分类体系,并与先前方案进行比较。结果显示,部分既有资源可能严重欠标注;虽然桥接类型分类的标注者间一致性达到中等水平,但对桥接实例的全面识别重合度较低,多数分歧源于对涉及实体的主观理解差异。
原文摘要 · Abstract (English)
Bridging refers to the associative relationship between inferable entities in a discourse and the antecedents which allow us to understand them, such as understanding what "the door" means with respect to an aforementioned "house". As identifying associative relations between entities is an inherently subjective task, it is difficult to achieve consistent agreement in the annotation of bridging anaphora and their antecedents. In this paper, we explore the subjectivity involved in the annotation of bridging instances at three levels: anaphor recognition, antecedent resolution, and bridging subtype selection. To do this, we conduct an annotation pilot on the test set of the existing GUM corpus, and propose a newly developed classification system for bridging subtypes, which we compare to previously proposed schemes. Our results suggest that some previous resources are likely to be severely under-annotated. We also find that while agreement on the bridging subtype category was moderate, annotator overlap for exhaustively identifying instances of bridging is low, and that many disagreements resulted from subjective understanding of the entities involved.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。