arXiv:2505.14499cs.CLcs.AI2025-05中稿 · ICONIP2024被引 3

用大模型生成解释,提升小模型对图文情感的分析能力

Enhanced Multimodal Aspect-Based Sentiment Analysis by LLM-Generated Rationales

  • 让大模型生成理由,注入小模型增强理解
  • 在三个主流数据集上显著优于现有方法
  • 适合需要高精度图文情感分析的研究者

近年来,多模态方面级情感分析(MABSA)受到广泛关注。现有方法主要依赖预训练的小语言模型(SLMs)从图像和文本中提取与方面和情感相关的信息,以实现模态对齐。然而,小模型容量和知识有限,常导致对语义、方面、情感及其关联的识别不准确。相比之下,大语言模型(LLMs)在探索多模态数据细粒度信息方面表现优异。尽管有研究指出,LLMs在方面级情感分析(ABSA)领域仍不及微调后的小模型,但本工作提出一种新框架LRSA,将小模型的决策能力与大模型提供的额外信息结合。具体地,我们将大模型生成的解释作为理由注入小模型,并采用双交叉注意力机制强化特征交互与融合,从而提升小模型识别方面和情感的能力。在两个基线模型上进行大量实验,结果表明该方法在三个广泛使用的基准数据集上均表现出色,证明其通用性和对多数预训练模型的适用性。

原文摘要 · Abstract (English)

There has been growing interest in Multimodal Aspect-Based Sentiment Analysis (MABSA) in recent years. Existing methods predominantly rely on pre-trained small language models (SLMs) to collect information related to aspects and sentiments from both image and text, with an aim to align these two modalities. However, small SLMs possess limited capacity and knowledge, often resulting in inaccurate identification of meaning, aspects, sentiments, and their interconnections in textual and visual data. On the other hand, Large language models (LLMs) have shown exceptional capabilities in various tasks by effectively exploring fine-grained information in multimodal data. However, some studies indicate that LLMs still fall short compared to fine-tuned small models in the field of ABSA. Based on these findings, we propose a novel framework, termed LRSA, which combines the decision-making capabilities of SLMs with additional information provided by LLMs for MABSA. Specifically, we inject explanations generated by LLMs as rationales into SLMs and employ a dual cross-attention mechanism for enhancing feature interaction and fusion, thereby augmenting the SLMs' ability to identify aspects and sentiments. We evaluated our method using two baseline models, numerous experiments highlight the superiority of our approach on three widely-used benchmarks, indicating its generalizability and applicability to most pre-trained models for MABSA.

多模态情感分析大模型推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。