用低秩专家网络提升情感与情绪分析的多任务学习效果
Multimodal Mixture of Low-Rank Experts for Sentiment Analysis and Emotion Recognition
- 设计共享与任务专属专家，分离共性与特性特征
- 在CMU-MOSI上达当前最优，CMU-MOSEI表现优异
- 适合需要高效多模态情感识别的研究者
多任务学习（MTL）可高效迁移其他任务获得的知识。由于多模态情感分析（MSA）与多模态情绪识别（MER）具有高度相关性，适合联合训练。然而，现有方法主要采用硬参数共享，忽视了复杂任务关联带来的参数冲突。本文提出一种新的MTL方法——多模态低秩专家混合模型（MMoLRE），通过共享专家和任务特定专家分别建模公共与独特特征，避免参数冲突。同时，借鉴混合专家（MoE）框架中的低秩结构，设计低秩专家网络，在专家数量增加时仍保持较低参数与计算开销。在CMU-MOSI和CMU-MOSEI数据集上的大量实验表明，MMoLRE在MSA任务上达到当前最优性能，在MER任务上表现竞争力。
原文摘要 · Abstract (English)
Multi-task learning (MTL) enables the efficient transfer of extra knowledge acquired from other tasks. The high correlation between multimodal sentiment analysis (MSA) and multimodal emotion recognition (MER) supports their joint training. However, existing methods primarily employ hard parameter sharing, ignoring parameter conflicts caused by complex task correlations. In this paper, we present a novel MTL method for MSA and MER, termed Multimodal Mixture of Low-Rank Experts (MMoLRE). MMoLRE utilizes shared and task-specific experts to distinctly model common and unique task characteristics, thereby avoiding parameter conflicts. Additionally, inspired by low-rank structures in the Mixture of Experts (MoE) framework, we design low-rank expert networks to reduce parameter and computational overhead as the number of experts increases. Extensive experiments on the CMU-MOSI and CMU-MOSEI benchmarks demonstrate that MMoLRE achieves state-of-the-art performance on the MSA task and competitive results on the MER task.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。