arXiv:2412.00763cs.CLcs.AI2024-12被引 10

提出PGSO模型,优化情感分析中的长文本依赖关系建模

PGSO: Prompt-based Generative Sequence Optimization Network for Aspect-based Sentiment Analysis

  • 设计动态序列优化机制,根据词序得分重排上下文
  • 在多个基准上平均提升3.52% F1得分,优于现有方法
  • 适合处理长文本中细粒度的情感依存关系分析

近年来,基于生成式预训练的模型在方面级情感分析(ABSA)任务中表现优异。然而,以往工作过度关注通过多样化模板改写训练目标以提升解码效果,忽视了生成模型内部的优化。尽管目标导向优化方法取得显著成果,但在处理复杂长文本时仍面临挑战,因生成模型的位置编码机制难以捕捉隐含的长距离依赖关系(如方面-观点关系)。为此,本文首先分析问题成因,提出两种序列优化策略:基于规则的静态优化与基于得分的动态优化。前者利用手工设计的依赖优先级重排上下文,后者通过神经网络计算词序得分动态调节上下文顺序。在此基础上,提出统一的提示驱动生成序列优化网络(PGSO),联合优化训练目标与生成模型。PGSO包含提示构建与序列调节两个组件:前者基于无监督训练对象构建任务特定提示,充分调动预训练模型能力;后者融合语义、句法与原始序列信息,动态调节上下文顺序。在四个ABSA任务及多个基准上的实验表明,PGSO相较现有最优方法平均提升3.52% F1得分。

原文摘要 · Abstract (English)

Recently, generative pre-training based models have demonstrated remarkable results on Aspect-based Sentiment Analysis (ABSA) task. However, previous works overemphasize crafting various templates to paraphrase training targets for enhanced decoding, ignoring the internal optimizations on generative models. Despite notable results achieved by these target-oriented optimization methods, they struggle with the complicated long texts since the implicit long-distance relation, e.g., aspect-opinion relation, is difficult to extract under the position embedding mechanism in generative models. Thus, in this paper, we first clarify the causes of the problem and introduce two sequence optimization strategies: the rule-based static optimization and the score-based dynamic optimization. The rule-based approach relies on handcraft priority of dependency relation to reorder the context, while the score-based algorithm dynamically regulates the contextual sequence by calculating word position scores using neural network. Based on the dynamic optimization structure, we further propose a unified Prompt-based Generative Sequence Optimization network (named PGSO), which jointly optimizes the training target as well as the generative model. Specifically, PGSO contains two components, namely, prompt construction and sequence regulator. The former constructs a task-specific prompt based on unsupervised training objects to fully utilize the pre-trained model. The latter jointly leverages semantic, syntactic and original-sequence information to dynamically regulate contextual sequence. Our experiments conducted on four ABSA tasks across multiple benchmarks indicate that PGSO outperforms state-of-the-art methods, with an average improvement of 3.52% in F1 score.

情感分析序列优化生成模型提示学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。