利用ReLU网络的几何特性生成精准因果解释,无需牺牲模型性能。
Causal Explanations from the Geometric Properties of ReLU Neural Networks
- 基于ReLU网络的分段线性几何结构生成解释
- 直接从网络几何提取规则,保持原模型行为一致性
- 适合需要高可信度解释的自动驾驶等安全关键场景
神经网络在自主系统控制策略学习中表现高效,但其黑箱特性导致决策过程难以理解,给安全性保障带来挑战。可解释人工智能(XAI)与可解释强化学习(XRL)旨在解析神经网络及自主代理的决策机制,其中因果解释关注于提供“为何如此”和“为何不那样”的回答。然而,现有方法多依赖简化版模型,虽可解释,但性能显著下降,且未必准确反映原始模型的决策逻辑,无法保证安全性。最新研究揭示,ReLU网络可视为由n维凸多面体划分的分段线性函数。通过这一几何视角,网络将输入空间划分为多个区域,每个区域对输出神经元应用单一线性变换。本文证明,可直接利用该几何结构生成因果解释,相较于以往方法更精确反映网络行为,且无需降级模型性能。
原文摘要 · Abstract (English)
Neural networks have proved an effective means of learning control policies for autonomous systems, but these learned policies are difficult to understand due to the black-box nature of neural networks. This lack of interpretability makes safety assurance for such autonomous systems challenging. The fields of eXplainable Artificial Intelligence (XAI) and eXplainable Reinforcement Learning (XRL) aim to interpret the decision making processes of neural networks and autonomous agents, respectively. In particular, work on causal explanations aims to provide "why" and "why not" explanations for why a model made a given decision. However, most of the work on explainability to date utilises a distilled version of the original model. While this distilled policy is interpretable, it necessarily degrades in performance significantly when compared to the original model, and is not guaranteed to be an accurate reflection of the decision making processes in the original model and as such cannot be used to guarantee its safety. Recent work on understanding the geometry of ReLU neural networks shows that a ReLU network corresponds to a piecewise linear function divided into regions defined by an n-dimensional convex polytope. Through this lens, a neural network can be understood as dividing the input space into distinct regions which apply a single linear function for each output neuron. We show that this geometric representation can be used to generate causal explanations for the network's behaviour similar to previous work, but which extracts rules directly from the geometry of Neural Networks with the ReLU activation function, and is therefore an accurate reflection of the network's behaviour.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。