用Transformer改进表格标注法,提升情感三元组抽取效果
T-T: Table Transformer for Tagging-based Aspect Sentiment Triplet Extraction

- 将Transformer直接用于表格标注的关联学习,增强词间关系捕捉
- 在SemEval-2015和Laplace-Data数据集上达到新最优,计算成本更低
- 提出条带注意力机制,解决长序列和局部注意力不均问题
方面情感三元组抽取(ASTE)旨在从句子中提取由方面词、观点词和情感极性组成的三元组。表格标注法是一种主流方法,将句子编码为二维表格,实现任意两词间的标签关系。以往工作聚焦于设计下游关系学习模块以更好捕捉表中词元间的交互,表明更强的关系建模能力可带来更大提升。受此启发,本文尝试直接使用Transformer作为下游关系学习模块。由于Transformer具备强大的语义建模能力,预期能显著提升性能。然而,因表格长度与输入序列长度呈二次关系,直接使用Transformer面临两大挑战:表格序列过长、局部注意力交互不公。为此,提出新型表格式Transformer(T-T)。具体而言,引入带有循环移位策略的条带注意力机制:前者将全局注意力改为仅关注二维局部窗口,后者促进不同注意力窗口间的交互。大量实验表明,T-T作为下游关系学习模块,在多个基准数据集上取得当前最佳性能,且计算开销更低。
原文摘要 · Abstract (English)
Aspect sentiment triplet extraction (ASTE) aims to extract triplets composed of aspect terms, opinion terms, and sentiment polarities from given sentences. The table tagging method is a popular approach to addressing this task, which encodes a sentence into a 2-dimensional table, allowing for the tagging of relations between any two words. Previous efforts have focused on designing various downstream relation learning modules to better capture interactions between tokens in the table, revealing that a stronger capability to capture relations can lead to greater improvements in the model. Motivated by this, we attempt to directly utilize transformer layers as downstream relation learning modules. Due to the powerful semantic modeling capability of transformers, it is foreseeable that this will lead to excellent improvement. However, owing to the quadratic relation between the length of the table and the length of the input sentence sequence, using transformers directly faces two challenges: overly long table sequences and unfair local attention interaction. To address these challenges, we propose a novel Table-Transformer (T-T) for the tagging-based ASTE method. Specifically, we introduce a stripe attention mechanism with a loop-shift strategy to tackle these challenges. The former modifies the global attention mechanism to only attend to a 2-dimensional local attention window, while the latter facilitates interaction between different attention windows. Extensive and comprehensive experiments demonstrate that the T-T, as a downstream relation learning module, achieves state-of-the-art performance with lower computational costs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。