arXiv:2509.09160cs.CLcs.AI2025-09中稿 · the IEEE Internati…被引 1

针对图文情感分类中的文本偏见问题,提出反事实增强去偏框架提升模型鲁棒性。

Target-oriented Multimodal Sentiment Classification with Counterfactual-enhanced Debiasing

  • 通过反事实数据增强,微调情感相关因果特征以引导注意力
  • 在多个基准数据集上超越现有最佳方法,显著降低偏见影响
  • 适合需要高鲁棒性图文情感分析的场景,如社交媒体内容理解

目标导向的多模态情感分类旨在从图像-文本对中预测特定目标的情感极性。现有方法虽表现良好,但过度依赖文本内容,忽视数据集中的词级上下文偏见,导致文本特征与标签间产生虚假关联,降低分类精度。本文提出一种新型反事实增强去偏框架,以减少此类虚假关联。该框架引入反事实数据增强策略,仅轻微改变情感相关的因果特征,生成细节匹配的图文样本,引导模型关注与情感相关的内容。此外,为从反事实数据中学习鲁棒特征并促进模型决策,设计自适应去偏对比学习机制,有效缓解偏见词语的影响。在多个基准数据集上的实验结果表明,所提方法优于现有最先进基线。

原文摘要 · Abstract (English)

Target-oriented multimodal sentiment classification seeks to predict sentiment polarity for specific targets from image-text pairs. While existing works achieve competitive performance, they often over-rely on textual content and fail to consider dataset biases, in particular word-level contextual biases. This leads to spurious correlations between text features and output labels, impairing classification accuracy. In this paper, we introduce a novel counterfactual-enhanced debiasing framework to reduce such spurious correlations. Our framework incorporates a counterfactual data augmentation strategy that minimally alters sentiment-related causal features, generating detail-matched image-text samples to guide the model's attention toward content tied to sentiment. Furthermore, for learning robust features from counterfactual data and prompting model decisions, we introduce an adaptive debiasing contrastive learning mechanism, which effectively mitigates the influence of biased words. Experimental results on several benchmark datasets show that our proposed method outperforms state-of-the-art baselines.

多模态情感分析去偏

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。