考虑数据质量差异，提升低质多模态情感分析效果
Data Uncertainty-Aware Learning for Multimodal Aspect-based Sentiment Analysis
- 根据图像质量与跨模态相关性动态评估样本权重
- 在Twitter-2015上达到当前最好性能
- 适合处理噪声多、质量不一的真实场景数据
多模态方面级情感分析（MABSA）是一项细粒度任务，旨在识别文本-图像对中的方面级情感信息。然而，我们发现低质量样本（如低分辨率图像含噪声）的情感难以准确识别。现实中，样本间数据质量差异显著，这种不确定性称为数据不确定性。以往方法对不同质量样本同等对待，忽略了其影响。本文提出一种新的数据不确定性感知方法UA-MABSA，通过结合图像质量与方面级跨模态相关性，动态加权样本损失，使模型更关注高质量且具有挑战性的样本。大量实验表明，该方法在Twitter-2015数据集上达到当前最优性能，进一步分析验证了质量评估策略的有效性。
原文摘要 · Abstract (English)
As a fine-grained task, multimodal aspect-based sentiment analysis (MABSA) mainly focuses on identifying aspect-level sentiment information in the text-image pair. However, we observe that it is difficult to recognize the sentiment of aspects in low-quality samples, such as those with low-resolution images that tend to contain noise. And in the real world, the quality of data usually varies for different samples, such noise is called data uncertainty. But previous works for the MABSA task treat different quality samples with the same importance and ignored the influence of data uncertainty. In this paper, we propose a novel data uncertainty-aware multimodal aspect-based sentiment analysis approach, UA-MABSA, which weighted the loss of different samples by the data quality and difficulty. UA-MABSA adopts a novel quality assessment strategy that takes into account both the image quality and the aspect-based cross-modal relevance, thus enabling the model to pay more attention to high-quality and challenging samples. Extensive experiments show that our method achieves state-of-the-art (SOTA) performance on the Twitter-2015 dataset. Further analysis demonstrates the effectiveness of the quality assessment strategy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。