arXiv:2605.06643cs.CVcs.AI2026-05

首个统一基准测试揭示多模态领域泛化进展有限,多数方法实际提升微弱。

Are We Making Progress in Multimodal Domain Generalization? A Comprehensive Benchmark Study

论文配图:Are We Making Progress in Multimodal Domain Generalization? A Comprehensive Benchmark Study
图 1 · 摘自论文原文
  • 构建跨6数据集、3任务的标准化评估框架,统一比较条件。
  • 实测显示新方法仅小幅超越基础模型,且无通用最优解。
  • 暴露模型在噪声与缺失模态下严重退化,可信度存疑。

尽管多模态领域泛化(MMDG)日益流行以提升模型鲁棒性,但现有性能提升是否反映真实算法进步仍不明确。当前研究分散,数据集、模态配置与实验设置差异大,且基准多聚焦动作识别,忽视输入干扰、模态缺失和模型可信度等现实挑战。为解决此问题,我们提出MMDG-Bench,首个统一且全面的MMDG基准,涵盖6个数据集、3类任务(动作识别、机械故障诊断、情感分析),支持6种模态组合、9种代表性方法及多种评估设置。除标准准确率外,还系统评估了抗干扰能力、缺失模态泛化、误分类检测与分布外检测。共训练7,402个神经网络,在95个跨域任务中得出五项发现:(1) 公平比较下,近期专用方法仅小幅优于ERM基线;(2) 无方法在所有数据集或模态组合上持续领先;(3) 上界性能仍有显著差距,表明该问题远未解决;(4) 三模态融合并未始终优于最强双模态配置;(5) 所有方法在扰动与模态缺失下均出现显著退化,部分进一步损害模型可信度。

原文摘要 · Abstract (English)

Despite the growing popularity of Multimodal Domain Generalization (MMDG) for enhancing model robustness, it remains unclear whether reported performance gains reflect genuine algorithmic progress or are artifacts of inconsistent evaluation protocols. Current research is fragmented, with studies varying significantly across datasets, modality configurations, and experimental settings. Furthermore, existing benchmarks focus predominantly on action recognition, often neglecting critical real-world challenges such as input corruptions, missing modalities, and model trustworthiness. This lack of standardization obscures a reliable assessment of the field's advancement. To address this issue, we introduce MMDG-Bench, the first unified and comprehensive benchmark for MMDG, which standardizes evaluation across six datasets spanning three diverse tasks: action recognition, mechanical fault diagnosis, and sentiment analysis. MMDG-Bench encompasses six modality combinations, nine representative methods, and multiple evaluation settings. Beyond standard accuracy, it systematically assesses corruption robustness, missing-modality generalization, misclassification detection, and out-of-distribution detection. With 7, 402 neural networks trained in total across 95 unique cross-domain tasks, MMDG-Bench yields five key findings: (1) under fair comparisons, recent specialized MMDG methods offer only marginal improvements over ERM baseline; (2) no single method consistently outperforms others across datasets or modality combinations; (3) a substantial gap to upper-bound performance persists, indicating that MMDG remains far from solved; (4) trimodal fusion does not consistently outperform the strongest bimodal configurations; and (5) all evaluated methods exhibit significant degradation under corruption and missing-modality scenarios, with some methods further compromising model trustworthiness.

多模态领域泛化基准测试鲁棒性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。