动态调整提示词,让多模态模型更准地判断情感
MAF: Multimodal Adaptive Few-shot Prompting for Sentiment Analysis with MLLMs

- 根据输入内容自动找最相关的示例来优化提示
- 在多个数据集上显著提升情感分析准确率
- 适合需要精准情感理解的多模态应用
多模态大语言模型(MLLMs)在理解复杂多模态内容方面表现出色,但在情感分析任务中对提示设计极为敏感,静态统一的提示难以捕捉不同输入中的细微多模态线索。为此,我们提出多模态自适应少样本提示框架(MAF),通过动态检索与查询相关的示例,以情境敏感的方式激发MLLM的情感推理能力。MAF构建了融合面部表情、场景上下文和文本语义的演示检索模块,并引入唇部运动幅度检测机制,实现多人场景下的精准说话人识别。不同于传统固定权重融合,其轻量级系数生成网络可实时输出查询相关的融合权重,对多模态相似度得分进行加权聚合,从而检索出前K个最具信息量的示例。通过多次生成候选结果并进行多数投票,进一步提升预测稳定性。在多个公开基准数据集上的大量实验表明,MAF在性能上显著优于基线模型,且保持与强大多模态情感分析方法相当的竞争力。
原文摘要 · Abstract (English)
Multimodal large language models (MLLMs) have demonstrated remarkable capabilities in understanding complex multimodal content. However, their performance in sentiment analysis exhibits acute sensitivity to prompt design, rendering static, uniformly applied prompts inherently suboptimal for capturing the nuanced multimodal cues that vary across inputs. To address this limitation, we propose a Multimodal Adaptive Few-Shot Prompting (MAF) framework, which dynamically retrieves and integrates query-relevant demonstrations to elicit the sentiment reasoning capabilities of MLLMs in a context-sensitive manner. MAF constructs a demonstration retrieval module that holistically encodes facial expressions, scene context, and textual semantics, with a lip movement amplitude detection mechanism introduced for accurate speaker identification in multi-person scenarios. Departing from conventional fixed-weight fusion, a lightweight coefficient generation network is trained to output query-conditioned fusion weights in real time, enabling weighted aggregation of multimodal similarity scores to retrieve the top-K most informative demonstrations. Prediction stability is further enhanced through majority voting over multiple candidate outputs generated by the MLLM. Extensive experiments on public benchmark datasets demonstrate that MAF achieves substantial and consistent performance improvements over the corresponding backbone variants and remains competitive with strong multimodal sentiment-analysis baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。