用语义标签细化核心消解评估，看清模型在人、地、事件上的真实表现。
Interpretable Coreference Resolution Evaluation Using Explicit Semantics
- 将命名实体与概念识别结果叠加到消解输出，给指代项打语义标签
- 在OntoNotes等3个数据集上发现聚合指标掩盖的系统性弱点
- 诊断结果可指导低成本数据增强，提升跨领域性能
核心消解通常使用CoNLL-F1等整体统计指标评估，仅衡量预测与标准簇之间的结构重叠。这类指标诊断能力有限，无法揭示系统在人物、地点或事件等特定语义类别上的表现差异，难以指导改进。本文提出一种基于语义增强的评估框架：将概念与命名实体识别（CNER）应用于核心消解输出，为名词指代项分配语义标签，并传播至整个共指簇，从而计算按语义类别划分的细分得分。在OntoNotes、LitBank和PreCo数据集上的实验表明，该框架揭示了被整体指标掩盖的系统性缺陷。此外，这些诊断结果可用于设计针对性的低成本数据增强策略，实现可观的跨域性能提升。
原文摘要 · Abstract (English)
Coreference resolution is typically evaluated using aggregate statistical metrics such as CoNLL-F1, which measure structural overlap between predicted and gold clusters. While widely used, these metrics offer limited diagnostic insights, penalizing errors without revealing whether a system struggles with specific semantic categories, such as people, locations, or events, and making it difficult to interpret model capabilities or derive actionable improvements. We address this gap by introducing a semantically-enhanced evaluation framework for coreference resolution. Our approach overlays Concept and Named Entity Recognition (CNER) onto coreference outputs, assigning semantic labels to nominal mentions and propagating them to entire coreference clusters. This enables the computation of typed scores aimed at evaluating mention extraction and linking capabilities stratified by semantic class. Across our experiments on OntoNotes, LitBank, and PreCo, we show that our framework uncovers systematic weaknesses that remain obscured by aggregate metrics. Furthermore, we demonstrate that these diagnostics can be used to design targeted, low-cost data augmentation strategies, achieving measurable out-of-domain improvements.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。