提出多粒度路由与不确定性感知融合，提升多模态情感分析鲁棒性。
MRUF: Multi-granularity Routing with Uncertainty-Aware Fusion for Robust Multimodal Sentiment Analysis
- 通过多粒度路由与不确定性校准，动态评估各模态可靠性。
- 在CMU-MOSI/MOSEI上优于强基线，未对齐场景下提升显著。
- 适合处理噪声大、模态质量不稳定的实际情感分析任务。
多模态情感分析依赖语言、视觉和语音线索，但话语级模态质量可能受遮挡、背景噪声、运动模糊或转录不全影响，导致传统融合方法过度信任不可靠模态。我们提出MRUF，一种可靠性感知融合方法，结合多粒度路由与不确定性感知校准。MRUF总结情感相关表示，执行子空间级与模态级路由，并通过留一法误差增加监督模态路由，以估计话语级模态重要性。进一步预测模态级不确定性，通过反方差重加权优化模态门控，同时利用模态不变对比对齐稳定共享表示空间。在对齐与非对齐设置下的CMU-MOSI和CMU-MOSEI数据集上实验表明，持续优于强基线；机制分析验证：预测不确定性更高的模态获得更低的融合权重。
原文摘要 · Abstract (English)
Multimodal sentiment analysis relies on language, visual, and acoustic cues, but utterance-level modality quality may vary due to occlusion, background noise, motion blur, or imperfect transcripts, causing conventional fusion to over-trust unreliable modalities. We propose MRUF, a reliability-aware fusion method that combines multi-granularity routing with uncertainty-aware calibration. MRUF summarizes sentiment-relevant representations, performs subspace- and modality-level routing, and supervises modality routing with leave-one-out error increases to estimate utterance-level modality importance. It further predicts modality-wise uncertainty and refines modality gates through inverse-variance reweighting, while modality-invariant contrastive alignment stabilizes the shared representation space. Experiments on CMU-MOSI and CMU-MOSEI under aligned and unaligned settings show consistent improvements over strong baselines, and mechanism analysis verifies that modalities with higher predicted uncertainty receive lower fusion weights.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。