通过定制数据增强提升神经网络权重逆向还原能力
Data Augmentation Techniques to Reverse-Engineer Neural Network Weights from Input-Output Queries
- 设计专用于捕捉隐藏层表征空间的新增强方法
- 实现参数量达训练数据100倍的网络权重逆向还原
- 适合研究模型可逆性与对抗攻防的科研人员
在教师-学生框架下，可通过查询教师网络的输入输出映射来逆向推导其权重。理想情况下应使用教师网络的训练数据作为查询样本，但当教师参数量超过训练数据时，学生模型会过拟合于查询样本，难以对齐教师参数。本文探索数据增强技术以更有效地采样教师网络的输入输出映射，目标是激发其隐藏层中丰富的表示特征。发现旋转、翻转、加噪声等常规增强手段几乎无改善效果。为此，我们设计了针对网络隐藏层表征空间优化的新增强策略，显著扩展了当前可逆向还原的网络规模。实验表明，该方法可成功恢复参数量为训练数据点100倍的网络。
原文摘要 · Abstract (English)
Network weights can be reverse-engineered given enough informative samples of a network's input-output function. In a teacher-student setup, this translates into collecting a dataset of the teacher mapping -- querying the teacher -- and fitting a student to imitate such mapping. A sensible choice of queries is the dataset the teacher is trained on. But current methods fail when the teacher parameters are more numerous than the training data, because the student overfits to the queries instead of aligning its parameters to the teacher. In this work, we explore augmentation techniques to best sample the input-output mapping of a teacher network, with the goal of eliciting a rich set of representations from the teacher hidden layers. We discover that standard augmentations such as rotation, flipping, and adding noise, bring little to no improvement to the identification problem. We design new data augmentation techniques tailored to better sample the representational space of the network's hidden layers. With our augmentations we extend the state-of-the-art range of recoverable network sizes. To test their scalability, we show that we can recover networks of up to 100 times more parameters than training data-points.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。