arXiv:2502.12209stat.MLcs.AI2025-02

提出更准确的特征重要性解释方法,提升模型可解释性。

Suboptimal Shapley Value Explanations

  • 基于不确定性重加权,优化特征缺失基线选择
  • 新方法使解释结果与人类理解更一致,显著提升一致性
  • 适用于NLP任务中深度模型的可解释性分析

深度神经网络在多种应用中表现出强大能力。Shapley值已成为分析特征重要性的主流工具,用于理解深度模型的推理过程。计算Shapley值需选择代表特征缺失的基线,但现有随机和条件基线可能对解释产生负面影响。本文通过分析不同基线的次优性,识别出存在方向性偏差的基线:当某特征被替换为$m{x}'_i$时,其与其他特征的非对称交互会显著偏向模型输出。研究发现,$p(y|m{x}'_i) = p(y)$ 可能最小化该非对称交互。进一步将$m{x}'_i$对标签空间$L$的无信息性推广,避免估计$p(y)$,设计了一种简单的基于不确定性的重加权机制,加速计算。在多个NLP任务上进行实验,定量分析验证了该机制的有效性。同时,通过衡量可解释方法生成的解释与人类理解的一致性,揭示了模型推理与人类认知之间的差异。

原文摘要 · Abstract (English)

Deep Neural Networks (DNNs) have demonstrated strong capacity in supporting a wide variety of applications. Shapley value has emerged as a prominent tool to analyze feature importance to help people understand the inference process of deep neural models. Computing Shapley value function requires choosing a baseline to represent feature's missingness. However, existing random and conditional baselines could negatively influence the explanation. In this paper, by analyzing the suboptimality of different baselines, we identify the problematic baseline where the asymmetric interaction between $\bm{x}'_i$ (the replacement of the faithful influential feature) and other features has significant directional bias toward the model's output, and conclude that $p(y|\bm{x}'_i) = p(y)$ potentially minimizes the asymmetric interaction involving $\bm{x}'_i$. We further generalize the uninformativeness of $\bm{x}'_i$ toward the label space $L$ to avoid estimating $p(y)$ and design a simple uncertainty-based reweighting mechanism to accelerate the computation process. We conduct experiments on various NLP tasks and our quantitative analysis demonstrates the effectiveness of the proposed uncertainty-based reweighting mechanism. Furthermore, by measuring the consistency of explanations generated by explainable methods and human, we highlight the disparity between model inference and human understanding.

可解释性Shapley值NLP

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。