通过激活值分析神经网络决策，提升可解释性。
Feature-Guided Analysis of Neural Networks: A Replication Study
- 基于神经元激活值提取与任务相关的网络片段。
- 在MNIST和LSC数据集上，精确率高于已有研究。
- 模型架构与特征选择影响召回率，对精确率影响小。
理解神经网络为何做出特定决策，对安全关键应用至关重要。特征引导分析（FGA）提取与任务相关的神经网络片段。现有方法通常监控神经元激活值以提取相关规则。初步结果表明该方案可行，通过在两个试点案例中评估精度和召回率验证。然而，其在工业场景中的适用性仍需更多实证支持。为弥补这一不足，本文在由MNIST和LSC数据集组成的基准上评估了FGA的适用性。结果表明，FGA在生成解释神经网络行为规则方面有效，且在本研究基准上的精确率高于文献报道结果。我们还评估了神经网络架构、训练方式及特征选择对FGA有效性的影响，结果显示这些因素显著影响召回率，但对精确率影响微乎其微。
原文摘要 · Abstract (English)
Understanding why neural networks make certain decisions is pivotal for their use in safety-critical applications. Feature-Guided Analysis (FGA) extracts slices of neural networks relevant to their tasks. Existing feature-guided approaches typically monitor the activation of the neural network neurons to extract the relevant rules. Preliminary results are encouraging and demonstrate the feasibility of this solution by assessing the precision and recall of Feature-Guided Analysis on two pilot case studies. However, the applicability in industrial contexts needs additional empirical evidence. To mitigate this need, this paper assesses the applicability of FGA on a benchmark made by the MNIST and LSC datasets. We assessed the effectiveness of FGA in computing rules that explain the behavior of the neural network. Our results show that FGA has a higher precision on our benchmark than the results from the literature. We also evaluated how the selection of the neural network architecture, training, and feature selection affect the effectiveness of FGA. Our results show that the selection significantly affects the recall of FGA, while it has a negligible impact on its precision.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。