为多模态模型提供带不确定性的可信贡献度评估
Uncertainty-Aware Multimodal Learning via Conformal Shapley Intervals
- 用谢帕利值与置信推断结合，生成每种模态的贡献区间
- 在多个数据集上仅依赖少量有效模态即达到优异性能
- 可证明最优性，适合需要可解释性的实际应用
多模态学习通过融合多种数据模态提升预测性能，但不同模态贡献不均且依赖具体数据，难以判断哪些模态真正有用及其可信程度。量化模态重要性并同时给出不确定性是实现可解释和可靠多模态学习的核心。本文提出共形谢帕利区间（conformal Shapley intervals），将谢帕利值与共形推断结合，为每个模态构建带有不确定性的贡献区间。基于这些区间，我们设计了一种具有可证明最优性的模态选择方法：在给定观测特征条件下，所选模态子集的性能接近最优子集。在多个数据集上的实验表明，该方法能提供有意义的不确定性量化，并在仅依赖少数有效模态的情况下实现强预测性能。
原文摘要 · Abstract (English)
Multimodal learning combines information from multiple data modalities to improve predictive performance. However, modalities often contribute unequally and in a data dependent way, making it unclear which data modalities are genuinely informative and to what extent their contributions can be trusted. Quantifying modality level importance together with uncertainty is therefore central to interpretable and reliable multimodal learning. We introduce conformal Shapley intervals, a framework that combines Shapley values with conformal inference to construct uncertainty-aware importance intervals for each modality. Building on these intervals, we propose a modality selection procedure with a provable optimality guarantee: conditional on the observed features, the selected subset of modalities achieves performance close to that of the optimal subset. We demonstrate the effectiveness of our approach on multiple datasets, showing that it provides meaningful uncertainty quantification and strong predictive performance while relying on only a small number of informative modalities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。