用动态路由胶囊网络提升句子关系抽取的表示能力。
Re-Representation in Sentential Relation Extraction with Sequence Routing Algorithm
- 引入动态路由胶囊机制实现句子关系的可变表示。
- 在Tacred等数据集上超越现有模型，但在Wikidata上表现不佳。
- 发现标签噪声是性能下降主因，提出‘重表示’作为新挑战。
句法关系抽取（Sentential Relation Extraction, RE）是自然语言处理中的关键任务。本文提出基于胶囊网络动态路由的句法关系抽取方法。实验表明，该方法在主流数据集Tacred、Tacredrev、Retacred和Conll04上优于当前最优模型。进一步分析发现，尽管在小规模数据集上表现优异，但在更大规模的Wikidata数据集上性能下降明显。我们识别出Wikidata标签存在噪声，是导致性能受限的重要原因。此外，本文提出‘重表示’（re-representation）概念，源自神经科学中大脑为提升匹配效率而改变表征的机制——例如在国王:王后::男人:女人的类比中，经重表示后头项（国王、男人）与尾项（王后、女人）间的相似性增强。实验证明，所提模型相比基线模型能更有效地实现重表示。因此，除标签噪声外，重表示能力也成为句法关系抽取的新挑战。
原文摘要 · Abstract (English)
Sentential relation extraction (RE) is an important task in natural language processing (NLP). In this paper we propose to do sentential RE with dynamic routing in capsules. We first show that the proposed approach outperform state of the art on common sentential relation extraction datasets Tacred, Tacredrev, Retacred, and Conll04. We then investigate potential reasons for its good performance on the mentioned datasets, and yet low performance on another similar, yet larger sentential RE dataset, Wikidata. As such, we identify noise in Wikidata labels as one of the reasons that can hinder performance. Additionally, we show associativity of better performance with better re-representation, a term from neuroscience referred to change of representation in human brain to improve the match at comparison time. As example, in the given analogous terms King:Queen::Man:Woman, at comparison time, and as a result of re-representation, the similarity between related head terms (King,Man), and tail terms (Queen,Woman) increases. As such, our observation show that our proposed model can do re-representation better than the vanilla model compared with. To that end, beside noise in the labels of the distantly supervised RE datasets, we propose re-representation as a challenge in sentential RE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。