用微调的LLaVA模型实现图文情感分类，效果优于现有方法。
LLaVAC: Fine-tuning LLaVA as a Multimodal Sentiment Classifier
- 设计结构化提示词，融合单模态与多模态标签进行微调。
- 在MVSA-Single数据集上三种处理流程均超越现有方法。
- 适合需要图文情感分析的开发者，代码已开源。
我们提出一种用于构建多模态情感分析分类器的方法LLaVAC。该方法通过微调大型语言与视觉助手（LLaVA），实现对图像和文本模态的情感标签预测。其核心在于设计包含单模态与多模态标签的结构化提示词，以指导LLaVA进行有效的情感分类。在MVSA-Single数据集上的实验表明，该方法在三种数据处理流程中均优于现有技术。相关实现已公开于https://github.com/tchayintr/llavac。
原文摘要 · Abstract (English)
We present LLaVAC, a method for constructing a classifier for multimodal sentiment analysis. This method leverages fine-tuning of the Large Language and Vision Assistant (LLaVA) to predict sentiment labels across both image and text modalities. Our approach involves designing a structured prompt that incorporates both unimodal and multimodal labels to fine-tune LLaVA, enabling it to perform sentiment classification effectively. Experiments on the MVSA-Single dataset demonstrate that LLaVAC outperforms existing methods in multimodal sentiment analysis across three data processing procedures. The implementation of LLaVAC is publicly available at https://github.com/tchayintr/llavac.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。