用小模型筛选高置信样本,大模型只处理难例,兼顾准确率与效率。
Uncertainty-Aware Collaborative System of Large and Small Models for Multimodal Sentiment Analysis
- 小模型先筛,大模型只处理不确定样本,减少计算量。
- 将连续情感值转为分类任务,精准计算不确定性。
- 适合资源有限但需高精度情感分析的场景。
多模态大语言模型(MLLM)显著提升了多模态情感分析(MSA)性能,但其庞大的参数量导致训练与推理资源消耗巨大,严重制约效率。本文提出一种新型不确定性感知协作系统(U-ACS),融合不确定性感知基线模型(UBM)与MLLM。U-ACS分三阶段运行:首先,所有样本由UBM处理,保留高置信样本,低置信样本转入MLLM;为解决回归任务连续输出难以计算不确定性的难题,创新性地将连续情感标签预测转为分类任务,实现更精确的熵与不确定性计算。其次,MLLM进行初步处理:高置信样本或预测极性与UBM一致的低置信样本视为合格,其余样本继续传递。最后,对剩余低置信样本,利用前轮预测结果作为提示参考,进行二次推理。通过三阶段结果聚合,U-ACS在保持高情感分析准确率的同时,大幅提高效率,显著降低计算开销与资源消耗。大量实验验证了该方法在性能与效率间的优异平衡。
原文摘要 · Abstract (English)
Multimodal Large Language Models (MLLMs) have notably enhanced the performance of Multimodal Sentiment Analysis (MSA), yet their massive parameter scale leads to excessive resource consumption in training and inference, severely limiting model efficiency. To balance performance and efficiency for MSA, this paper innovatively proposes a novel Uncertainty-Aware Collaborative System (U-ACS) that integrates Uncertainty-aware Baseline Model (UBM) with MLLMs. U-ACS operates in three stages: First, all samples are processed by the UBM, retain high-confidence samples and forward low-confidence samples to the MLLM. Notably, to address the challenge that continuous outputs of regression tasks hinder uncertainty calculation, we innovatively convert the continuous sentiment label prediction task to a classification task, enabling a more accurate calculation of entropy and uncertainty. Second, the MLLM performs initial process. In this stage, high-confidence samples or low-confidence samples whose predictive sentiment polarity matches that of the UBM are deemed acceptable, while unqualified samples are forwarded for further processing. Finally, the MLLM performs secondary inference on remaining low-confidence samples using prompts augmented with prior rounds predictions as references. By aggregating results from the three stages, U-ACS preserves high MSA prediction accuracy while drastically boosting efficiency via offloading most simple samples to the UBM and minimizing MLLM processing volume. Extensive experiments verify that U-ACS maintains superior performance while significantly reducing computational overhead and resource consumption.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。