提出可高效重建训练数据的算法,且在有限网络宽度下有理论保证。
Efficient Techniques for Data Reconstruction, with Finite-Width Recovery Guarantees

- 基于初/终参数统一建模,结合随机特征模型推导出可证伪结果。
- 低维数据子空间下,所需网络宽度与子空间维数相关而非原始维度。
- 仅用首层权重变化估计子空间,降低搜索空间,适合实际应用。
针对神经网络训练数据的重构攻击严重威胁隐私,尤其当训练数据含敏感信息时。本文提出一种基于初始与训练后参数的统一优化框架,整合现有先进方法。在随机特征模型中,若网络宽度足够大,该框架可高概率成功重构训练数据,首次实现有限宽度下的可证明恢复效果,采用PAC风格的边界。当数据位于低维子空间时,成功重构所需的网络宽度可显著降低,其界限依赖于子空间维数而非环境维数。对于一般神经网络及未知数据方向的情况,我们设计了一种高效重构算法:通过训练过程中第一层权重的变化估计低维数据子空间,并仅使用最后一层权重进行重构,从而大幅缩减搜索空间并降低对网络宽度的要求。在合成数据集和CIFAR-10上的实验表明,该子空间感知方法优于传统全空间技术。
原文摘要 · Abstract (English)
Data reconstruction attacks on trained neural networks aim to recover the data on which the network has been trained and pose a significant threat to privacy, especially if the training dataset contains sensitive information. Here, we propose a unified optimization formulation of the data reconstruction problem based on initial and trained parameter values, incorporating state-of-the-art proposals. We show that in the random feature model, this formulation provably leads to training data reconstruction with high probability, provided the network width is sufficiently large; this unprecedented finite-width result uses PAC-style bounds. Furthermore, when the data lies in a low-dimensional subspace, we show that the network width requirement for successful reconstruction can be relaxed, with bounds depending on the subspace dimension rather than the ambient dimension. For general neural network models and unknown data orientations, we propose an efficient reconstruction algorithm that approximates the low-dimensional data subspace through the change in the first-layer weights during training and uses only the last-layer weights for reconstruction, thus reducing the search space dimension and the required network width for high-quality reconstructions. Our numerical experiments on synthetic datasets and CIFAR-10 confirm that our subspace-aware reconstruction approach outperforms standard full-space techniques.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。