用数学新框架量化数据与训练集的感知相似性
Hidden Activations Are Not Enough: A General Approach to Neural Network Predictions
- 基于量子表示理论构建网络感知的矩阵表示
- 在多个MLP模型上检测对抗样本效果优于传统方法
- 适用于任意架构与任务，适合研究模型泛化性
我们提出一种基于量子表示理论的新型数学分析框架，用于量化神经网络对新数据样本与训练数据之间的感知相似性。通过数据样本诱导的量子表示，我们捕捉的信息比传统隐藏层输出更丰富。该表示将前向传播的复杂计算抽象为单一矩阵，使我们在矩阵空间中能运用简单的几何与统计分析来研究网络预测。研究成果不依赖具体网络结构或任务类型，具有广泛适用性。作为概念验证实验，我们在MNIST和FashionMNIST数据集上，针对多种MLP架构及攻击方法，成功检测对抗样本。代码已公开于开源仓库。
原文摘要 · Abstract (English)
We introduce a novel mathematical framework for analyzing neural networks using tools from quiver representation theory. This framework enables us to quantify the similarity between a new data sample and the training data, as perceived by the neural network. By leveraging the induced quiver representation of a data sample, we capture more information than traditional hidden layer outputs. This quiver representation abstracts away the complexity of the computations of the forward pass into a single matrix, allowing us to employ simple geometric and statistical arguments in a matrix space to study neural network predictions. Our mathematical results are architecture-agnostic and task-agnostic, making them broadly applicable. As proof of concept experiments, we apply our results for the MNIST and FashionMNIST datasets on the problem of detecting adversarial examples on different MLP architectures and several adversarial attack methods. Our experiments can be reproduced with our \href{https://github.com/MarcoArmenta/Hidden-Activations-are-not-Enough}{publicly available repository}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。