arXiv:2411.17777cs.LGcs.CV2024-11

通过生成器逆向还原神经网络输入,提升模型可解释性。

Network Inversion and Its Applications

  • 用条件生成器学习输入数据分布,重建导致特定输出的输入
  • 通过最小化特征余弦相似度和正交性约束,提升生成图像多样性
  • 适用于模型解释、异常检测和训练数据还原,实用性强

神经网络在各类应用中表现强大,但其决策过程常被视为“黑箱”,影响可解释性与可靠性,尤其在安全关键场景下。网络逆向技术可帮助窥探黑箱内部,揭示网络决策所依赖的特征与模式。本文提出一种简单而有效的方法:利用精心设计的生成器学习神经网络输入空间的数据分布,重建能产生目标输出的输入。为捕捉同一输出下的输入多样性,不直接传递条件标签,而是将其编码为向量与中间矩阵,并最小化生成图像特征间的余弦相似度。同时引入特征正交性作为正则项,惩罚特征格拉姆矩阵偏离单位矩阵的程度,确保特征正交性,促进不同标签间表示的独立性。论文进一步探讨了该方法在可解释性、分布外检测及训练数据重构中的应用。

原文摘要 · Abstract (English)

Neural networks have emerged as powerful tools across various applications, yet their decision-making process often remains opaque, leading to them being perceived as "black boxes." This opacity raises concerns about their interpretability and reliability, especially in safety-critical scenarios. Network inversion techniques offer a solution by allowing us to peek inside these black boxes, revealing the features and patterns learned by the networks behind their decision-making processes and thereby provide valuable insights into how neural networks arrive at their conclusions, making them more interpretable and trustworthy. This paper presents a simple yet effective approach to network inversion using a meticulously conditioned generator that learns the data distribution in the input space of the trained neural network, enabling the reconstruction of inputs that would most likely lead to the desired outputs. To capture the diversity in the input space for a given output, instead of simply revealing the conditioning labels to the generator, we encode the conditioning label information into vectors and intermediate matrices and further minimize the cosine similarity between features of the generated images. Additionally, we incorporate feature orthogonality as a regularization term to boost image diversity which penalises the deviations of the Gram matrix of the features from the identity matrix, ensuring orthogonality and promoting distinct, non-redundant representations for each label. The paper concludes by exploring immediate applications of the proposed network inversion approach in interpretability, out-of-distribution detection, and training data reconstruction.

网络逆向可解释性生成模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。