arXiv:2604.03650cs.CL2026-04

用Mamba建模对话情感演变,高效融合多模态信息

CAGMamba: Context-Aware Gated Cross-Modal Mamba Network for Multimodal Sentiment Analysis

  • 将上下文与当前语句构造成时序二元序列,显式建模情感演化
  • 通过可学习门控机制平衡跨模态融合与模态保留,提升融合可控性
  • 在三个基准数据集上达到领先或相当性能,适合对话情感分析研究者

多模态情感分析需有效建模跨模态交互与上下文依赖,同时保持计算高效。现有融合方法主要依赖Transformer-based跨模态注意力,其复杂度随序列长度呈二次增长,限制了可扩展性。此外,前序话语的上下文信息常通过拼接或独立融合方式引入,缺乏显式的时间建模以捕捉对话轮次间的情感演变。为此,我们提出CAGMamba:一种面向对话情感分析的上下文感知门控跨模态Mamba框架。具体地,我们将上下文特征与当前话语特征组织成时间有序的二元序列,为Mamba提供显式时序结构以建模情感演化。为进一步实现可控的跨模态融合,我们设计了门控跨模态Mamba网络(GCMN),通过可学习门控机制整合跨模态与单模态路径,平衡信息融合与模态保留,并采用三分支多任务目标(文本、音频、融合预测)进行训练。在三个基准数据集上的实验表明,CAGMamba在多个评估指标上达到最先进或具有竞争力的结果。代码已开源:https://github.com/User2024-xj/CAGMamba。

原文摘要 · Abstract (English)

Multimodal Sentiment Analysis (MSA) requires effective modeling of cross-modal interactions and contextual dependencies while remaining computationally efficient. Existing fusion approaches predominantly rely on Transformer-based cross-modal attention, which incurs quadratic complexity with respect to sequence length and limits scalability. Moreover, contextual information from preceding utterances is often incorporated through concatenation or independent fusion, without explicit temporal modeling that captures sentiment evolution across dialogue turns. To address these limitations, we propose CAGMamba, a context-aware gated cross-modal Mamba framework for dialogue-based sentiment analysis. Specifically, we organize the contextual and the current-utterance features into a temporally ordered binary sequence, which provides Mamba with explicit temporal structure for modeling sentiment evolution. To further enable controllable cross-modal integration, we propose a Gated Cross-Modal Mamba Network (GCMN) that integrates cross-modal and unimodal paths via learnable gating to balance information fusion and modality preservation, and is trained with a three-branch multi-task objective over text, audio, and fused predictions. Experiments on three benchmark datasets demonstrate that CAGMamba achieves state-of-the-art or competitive results across multiple evaluation metrics. All codes are available at https://github.com/User2024-xj/CAGMamba.

多模态情感分析Mamba对话建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。