用一张矩阵解释神经网络决策，让黑箱变透明
OMENN: One Matrix to Explain Neural Networks
- 将神经网络每层输出映射为线性变换矩阵，生成可解释的输入-特征关联图
- 在多个模型上实现局部精确归因，效果媲美顶尖解释方法
- 适合需要理解模型推理过程的研究者和开发者
深度学习模型常被视为黑箱，其决策过程难以解释。这一问题推动了可解释人工智能（XAI）的发展，其中基于归因的方法如LRP和GradCAM被广泛应用，但依赖近似计算，精度有限。为此，我们提出一种新的后处理方法——OMENN（One Matrix to Explain Neural Networks），将神经网络对特定输入的处理过程表示为一个单一、可解释的矩阵。该矩阵通过一系列线性变换构建，反映输入在各层中的传播路径。结果表明，OMENN能为现代模型（包括ViTs和CNNs）提供局部精确的归因解释。我们基于动态线性性质进行了理论分析，并在两个XAI基准上进行了充分验证，结果显示其性能与当前最优方法相当。
原文摘要 · Abstract (English)
Deep Learning (DL) models are often black boxes, making their decision-making processes difficult to interpret. This lack of transparency has driven advancements in eXplainable Artificial Intelligence (XAI), a field dedicated to clarifying the reasoning behind DL model predictions. Among these, attribution-based methods such as LRP and GradCAM are widely used, though they rely on approximations that can be imprecise. To address these limitations, we introduce One Matrix to Explain Neural Networks (OMENN), a novel post-hoc method that represents a neural network as a single, interpretable matrix for each specific input. This matrix is constructed through a series of linear transformations that represent the processing of the input by each successive layer in the neural network. As a result, OMENN provides locally precise, attribution-based explanations of the input across various modern models, including ViTs and CNNs. We present a theoretical analysis of OMENN based on dynamic linearity property and validate its effectiveness with extensive tests on two XAI benchmarks, demonstrating that OMENN is competitive with state-of-the-art methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。