用博弈论解释多模态模型如何互动,让黑箱决策更可信。
MultiSHAP: A Shapley-Based Framework for Explaining Cross-Modal Interactions in Multimodal AI Models
- 基于谢尔普利指数,量化视觉与文本元素间的协同或抑制关系。
- 能解释单个样本的预测原因,并发现跨数据集的通用交互模式。
- 不依赖模型内部结构,适用于开源和闭源模型,实用性强。
多模态人工智能模型在融合视觉与语言信息的任务中表现卓越,但其'黑箱'特性阻碍了高风险场景中的应用。现有解释方法如注意力图、Grad-CAM仅提供粗粒度的跨模态关系线索,无法精确量化模态间的协同效应,且仅适用于可访问内部权重的开源模型。本文提出MultiSHAP,一种基于谢尔普利交互指数的模型无关解释框架,能够将多模态预测归因于细粒度视觉块与文本词元之间的成对交互,适用于开/闭源模型。该方法提供:(1) 实例级解释,揭示单个样本中模态间的协同与抑制效应——解释'为何对这个输入做出特定预测';(2) 数据集级解释,挖掘跨样本的通用交互模式——揭示'模型如何整合多模态信息'。在公开多模态基准上的实验表明,MultiSHAP能忠实捕捉跨模态推理机制;真实案例研究进一步验证其实用性。该框架可拓展至多于两模态,为复杂多模态模型提供通用解释方案。
原文摘要 · Abstract (English)
Multimodal AI models have achieved impressive performance in tasks that require integrating information from multiple modalities, such as vision and language. However, their "black-box" nature poses a major barrier to deployment in high-stakes applications where interpretability and trustworthiness are essential. How to explain cross-modal interactions in multimodal AI models remains a major challenge. While existing model explanation methods, such as attention map and Grad-CAM, offer coarse insights into cross-modal relationships, they cannot precisely quantify the synergistic effects between modalities, and are limited to open-source models with accessible internal weights. Here we introduce MultiSHAP, a model-agnostic interpretability framework that leverages the Shapley Interaction Index to attribute multimodal predictions to pairwise interactions between fine-grained visual and textual elements (such as image patches and text tokens), while being applicable to both open- and closed-source models. Our approach provides: (1) instance-level explanations that reveal synergistic and suppressive cross-modal effects for individual samples - "why the model makes a specific prediction on this input", and (2) dataset-level explanation that uncovers generalizable interaction patterns across samples - "how the model integrates information across modalities". Experiments on public multimodal benchmarks confirm that MultiSHAP faithfully captures cross-modal reasoning mechanisms, while real-world case studies demonstrate its practical utility. Our framework is extensible beyond two modalities, offering a general solution for interpreting complex multimodal AI models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。