用逻辑编程生成深度网络全局解释，揭示特征与隐藏节点作用
xDNN(ASP): Explanation Generation System for Deep Neural Networks powered by Answer Set Programming
- 基于答案集编程提取神经网络的逻辑规则，实现全局解释
- 在两个合成数据集上保持高预测准确率，且能分析特征重要性
- 可指导减少隐藏层节点数，适合模型优化与可解释性研究
可解释人工智能（xAI）近年受到广泛关注。深度神经网络因其“黑箱”特性，其可解释性成为研究热点。现有方法多聚焦输入输出关系，忽略网络结构。本文提出xDNN(ASP)，一种基于答案集编程的深度神经网络解释生成系统，可生成全局解释。给定模型与训练数据，该系统提取满足答案集语义的逻辑程序，在理想情况下，程序的答案集与网络的输入输出对一一对应。实验使用两个合成数据集验证：所提取逻辑程序不仅保持高预测准确率，还能揭示特征重要性及隐藏节点对预测的影响，进而指导减少隐藏层节点数量，实现网络优化。
原文摘要 · Abstract (English)
Explainable artificial intelligence (xAI) has gained significant attention in recent years. Among other things, explainablility for deep neural networks has been a topic of intensive research due to the meteoric rise in prominence of deep neural networks and their "black-box" nature. xAI approaches can be characterized along different dimensions such as their scope (global versus local explanations) or underlying methodologies (statistic-based versus rule-based strategies). Methods generating global explanations aim to provide reasoning process applicable to all possible output classes while local explanation methods focus only on a single, specific class. SHAP (SHapley Additive exPlanations), a well-known statistical technique, identifies important features of a network. Deep neural network rule extraction method constructs IF-THEN rules that link input conditions to a class. Another approach focuses on generating counterfactuals which help explain how small changes to an input can affect the model's predictions. However, these techniques primarily focus on the input-output relationship and thus neglect the structure of the network in explanation generation. In this work, we propose xDNN(ASP), an explanation generation system for deep neural networks that provides global explanations. Given a neural network model and its training data, xDNN(ASP) extracts a logic program under answer set semantics that-in the ideal case-represents the trained model, i.e., answer sets of the extracted program correspond one-to-one to input-output pairs of the network. We demonstrate experimentally, using two synthetic datasets, that not only the extracted logic program maintains a high-level of accuracy in the prediction task, but it also provides valuable information for the understanding of the model such as the importance of features as well as the impact of hidden nodes on the prediction. The latter can be used as a guide for reducing the number of nodes used in hidden layers, i.e., providing a means for optimizing the network.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。