让多模态情感分析适应信号质量连续变化，自动过滤噪声。
QA-MoE: Towards a Continuous Reliability Spectrum with Quality-Aware Mixture of Experts for Robust Multimodal Sentiment Analysis
- 用自监督不确定性量化各模态可靠性，动态分配专家处理
- 在多种信号缺失或退化下表现稳定，最高提升6.2%准确率
- 适合真实场景中信号质量不稳定的多模态任务
多模态情感分析旨在从文本、语音和视觉信号中推断人类情感。然而在真实场景中，多模态输入常受动态噪声或模态缺失影响。现有方法通常将这些缺陷视为离散情况或假设固定损坏比例，限制了对连续变化可靠性条件的适应能力。为此，我们首次提出连续可靠性谱，将模态缺失与质量退化统一建模。在此基础上，提出QA-MoE——一种质量感知的专家混合框架，通过自监督的随机不确定性量化模态可靠性，显式指导专家路由，从而抑制不可靠信号带来的误差传播，同时保留任务相关信息。大量实验表明，QA-MoE在多种退化场景下均达到竞争性或领先性能，并展现出出色的单检查点全场景适应能力。
原文摘要 · Abstract (English)
Multimodal Sentiment Analysis (MSA) aims to infer human sentiment from textual, acoustic, and visual signals. In real-world scenarios, however, multimodal inputs are often compromised by dynamic noise or modality missingness. Existing methods typically treat these imperfections as discrete cases or assume fixed corruption ratios, which limits their adaptability to continuously varying reliability conditions. To address this, we first introduce a Continuous Reliability Spectrum to unify missingness and quality degradation into a single framework. Building on this, we propose QA-MoE, a Quality-Aware Mixture-of-Experts framework that quantifies modality reliability via self-supervised aleatoric uncertainty. This mechanism explicitly guides expert routing, enabling the model to suppress error propagation from unreliable signals while preserving task-relevant information. Extensive experiments indicate that QA-MoE achieves competitive or state-of-the-art performance across diverse degradation scenarios and exhibits a promising One-Checkpoint-for-All property in practice.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。