基于词对齐的算法，让不同语义图更直观可比。
Word Alignment-Based Evaluation of Uniform Meaning Representations
- 利用语义图中已有的词对齐信息匹配节点
- 相比传统方法，识别出的差异更可解释
- 适合需要精准语义分析的研究者使用
句子语义的图表示比较与评估面临挑战，因为不同表示可能拥有不同数量的节点，且难以确定应如何对应节点。现有方法倾向于最大化节点关系和属性的F1分数，却忽略相似性是故意还是偶然；因此，节点属性值的不匹配无法用于详细错误分析。本文提出一种基于词对齐的节点匹配算法，可比较同一句子的多个统一语义表示（UMR），并利用UMR中固有的词对齐信息。与此前广泛使用的smatch方法对比，该方法对词对齐更敏感，使语义表示比较更直观、可解释，同时避免了smatch固有的NP难搜索问题。相关实现脚本已公开提供。
原文摘要 · Abstract (English)
Comparison and evaluation of graph-based representations of sentence meaning is a challenge because competing representations of the same sentence may have different number of nodes, and it is not obvious which nodes should be compared to each other. Existing approaches favor node mapping that maximizes $F_1$ score over node relations and attributes, regardless whether the similarity is intentional or accidental; consequently, the identified mismatches in values of node attributes are not useful for any detailed error analysis. We propose a node-matching algorithm that allows comparison of multiple Uniform Meaning Representations (UMR) of one sentence and that takes advantage of node-word alignments, inherently available in UMR. We compare it with previously used approaches, in particular smatch (the de-facto standard in AMR evaluation), and argue that sensitivity to word alignment makes the comparison of meaning representations more intuitive and interpretable, while avoiding the NP-hard search problem inherent in smatch. A script implementing the method is freely available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。