让多模态模型说出靠谱的解释，提升可信度
CAuSE: Decoding Multimodal Classifiers using Faithful Natural Language Explanation
- 通过因果干预训练生成符合模型决策逻辑的自然语言解释
- 在多个数据集和模型上验证，解释忠实度显著优于现有方法
- 适合需要可解释AI的医疗、金融等高风险场景
多模态分类器通常像黑箱一样难以理解。尽管已有多种解释技术，但能像自然语言一样直观易懂的却很少。为建立信任，解释必须真实反映模型内部决策过程，这称为忠实性。本文提出CAuSE（因果抽象下的模拟解释），一种可为任意预训练多模态分类器生成忠实自然语言解释的新框架。通过广泛实证评估，证明了CAuSE在不同数据集和模型间的泛化能力。理论上，我们证明了基于交换干预训练的CAuSE构成了底层分类器的因果抽象。进一步通过重新设计的因果忠实度度量进行验证，结果显示其性能优于其他方法。定性分析也支持其优势。我们还进行了详细错误分析，定位了失败案例。代码已公开于https://github.com/newcodevelop/CAuSE。
原文摘要 · Abstract (English)
Multimodal classifiers function as opaque black box models. While several techniques exist to interpret their predictions, very few of them are as intuitive and accessible as natural language explanations (NLEs). To build trust, such explanations must faithfully capture the classifier's internal decision making behavior, a property known as faithfulness. In this paper, we propose CAuSE (Causal Abstraction under Simulated Explanations), a novel framework to generate faithful NLEs for any pretrained multimodal classifier. We demonstrate that CAuSE generalizes across datasets and models through extensive empirical evaluations. Theoretically, we show that CAuSE, trained via interchange intervention, forms a causal abstraction of the underlying classifier. We further validate this through a redesigned metric for measuring causal faithfulness in multimodal settings. CAuSE surpasses other methods on this metric, with qualitative analysis reinforcing its advantages. We perform detailed error analysis to pinpoint the failure cases of CAuSE. For replicability, we make the codes available at https://github.com/newcodevelop/CAuSE
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。