arXiv:2410.05484cs.LGcs.AI2024-10被引 1

通过因果分析揭示神经网络决策机制,无需修改模型即可实现可信解释。

Neural Networks Decoded: Targeted and Robust Analysis of Neural Network Decisions via Causal Explanations and Reasoning

  • 基于因果推断,干预输入特征观察其传播路径
  • 构建分组的因果节点图,清晰展示各层影响机制
  • 生成反事实案例,发现模型偏见并解释误判

尽管深度神经网络(DNN)取得了巨大成功并被广泛应用,其决策过程仍高度不透明,严重阻碍了在关键场景中的信任建立。现有可解释性方法常产生不一致或过于简化的解释,或需改变模型结构而损害性能。本文提出TRACER,一种基于因果推断理论的新方法,可在不修改网络架构且不影响性能的前提下,估计支撑DNN决策的因果动态。该方法系统性地对输入特征进行干预,观察其如何通过网络传播,影响内部激活与最终输出,据此评估各特征重要性,并将功能相似的层聚类为因果节点,构建高层级因果图,提供结构化、可解释的网络决策视图。此外,TRACER通过生成反事实案例揭示潜在模型偏见,并为误分类提供对比性解释。在多种数据集上的全面评估表明,TRACER优于现有方法,且具备构建高度压缩但保持高精度模型的潜力,展现出在理解与优化DNN方面的双重应用价值。

原文摘要 · Abstract (English)

Despite their success and widespread adoption, the opaque nature of deep neural networks (DNNs) continues to hinder trust, especially in critical applications. Current interpretability solutions often yield inconsistent or oversimplified explanations, or require model changes that compromise performance. In this work, we introduce TRACER, a novel method grounded in causal inference theory designed to estimate the causal dynamics underpinning DNN decisions without altering their architecture or compromising their performance. Our approach systematically intervenes on input features to observe how specific changes propagate through the network, affecting internal activations and final outputs. Based on this analysis, we determine the importance of individual features, and construct a high-level causal map by grouping functionally similar layers into cohesive causal nodes, providing a structured and interpretable view of how different parts of the network influence the decisions. TRACER further enhances explainability by generating counterfactuals that reveal possible model biases and offer contrastive explanations for misclassifications. Through comprehensive evaluations across diverse datasets, we demonstrate TRACER's effectiveness over existing methods and show its potential for creating highly compressed yet accurate models, illustrating its dual versatility in both understanding and optimizing DNNs.

神经网络解释因果推理可解释性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。