让模型聚焦关键目标，提升立场识别准确率
Stanceformer: Target-Aware Transformer for Stance Detection
- 设计目标感知注意力矩阵，强化对目标词的关注
- 在3个数据集上表现优于基线，零样本场景也有效
- 适配多种BERT和大模型，可推广至情感分析等任务
立场检测旨在判断文本对特定主题或目标的立场。现有方法依赖通用Transformer模型，难以有效关注目标信息，导致使用或忽略目标时性能相近，削弱了任务意义。为此，我们提出Stanceformer，一种目标感知的Transformer模型，在训练和推理中增强对目标词的自注意力。具体地，设计了目标感知矩阵，提升目标词的注意力得分。我们在多个基于BERT的模型（包括顶尖模型与大语言模型）上验证该方法，在三个立场检测数据集及一个零样本数据集上均取得更优效果。该方法不仅显著提升性能，还能泛化至其他领域，如基于方面的情感分析。代码已开源。
原文摘要 · Abstract (English)
The task of Stance Detection involves discerning the stance expressed in a text towards a specific subject or target. Prior works have relied on existing transformer models that lack the capability to prioritize targets effectively. Consequently, these models yield similar performance regardless of whether we utilize or disregard target information, undermining the task's significance. To address this challenge, we introduce Stanceformer, a target-aware transformer model that incorporates enhanced attention towards the targets during both training and inference. Specifically, we design a \textit{Target Awareness} matrix that increases the self-attention scores assigned to the targets. We demonstrate the efficacy of the Stanceformer with various BERT-based models, including state-of-the-art models and Large Language Models (LLMs), and evaluate its performance across three stance detection datasets, alongside a zero-shot dataset. Our approach Stanceformer not only provides superior performance but also generalizes even to other domains, such as Aspect-based Sentiment Analysis. We make the code publicly available.\footnote{\scriptsize\url{https://github.com/kgarg8/Stanceformer}}
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。