揭示解释方法可能反向推导预测，而非真实决策过程。
Are We Merely Justifying Results ex Post Facto? Quantifying Explanatory Inversion in Post-Hoc Model Explanations
- 提出量化框架，检测解释是否依赖输出而非输入。
- 发现LIME、SHAP等方法在虚假关联下易出现解释倒置。
- 新方法RBP可有效降低解释倒置，提升解释可信度。
后验解释方法通过将预测归因于输入特征来提供解释。理想的解释应反映输入如何导致输出。然而，一个根本问题浮现：这些解释是否无意中反转了输入与输出的自然关系？即，解释是否在用输出反推预测，而非反映真实的决策过程？为此，本文提出解释倒置量化（IQ）框架，用于衡量解释对输出的依赖程度及其与真实输入-输出关系的偏差。在合成数据上，我们验证了如LIME和SHAP等广泛使用的方法在表格、图像和文本领域均易出现倒置，尤其在存在虚假相关时。进一步提出“复现-戳点”（RBP）方法，一种简单且模型无关的增强机制，通过前向扰动检验来改进解释。理论证明，RBP可在IQ框架下有效缓解解释倒置。实验表明，在合成数据上，RBP平均可使倒置程度降低1.8%，适用于主流后验解释方法与多种任务场景。
原文摘要 · Abstract (English)
Post-hoc explanation methods provide interpretation by attributing predictions to input features. Natural explanations are expected to interpret how the inputs lead to the predictions. Thus, a fundamental question arises: Do these explanations unintentionally reverse the natural relationship between inputs and outputs? Specifically, are the explanations rationalizing predictions from the output rather than reflecting the true decision process? To investigate such explanatory inversion, we propose Inversion Quantification (IQ), a framework that quantifies the degree to which explanations rely on outputs and deviate from faithful input-output relationships. Using the framework, we demonstrate on synthetic datasets that widely used methods such as LIME and SHAP are prone to such inversion, particularly in the presence of spurious correlations, across tabular, image, and text domains. Finally, we propose Reproduce-by-Poking (RBP), a simple and model-agnostic enhancement to post-hoc explanation methods that integrates forward perturbation checks. We further show that under the IQ framework, RBP theoretically guarantees the mitigation of explanatory inversion. Empirically, for example, on the synthesized data, RBP can reduce the inversion by 1.8% on average across iconic post-hoc explanation approaches and domains.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。