提出GReDP方法，在低噪声下保护训练隐私且不损失梯度信息
Training with Differential Privacy: A Gradient-Preserving Noise Reduction Approach with Provable Security
- 在频域计算梯度，设计新降噪机制
- 噪声规模仅为DPSGD的一半，保持完整梯度
- 理论与实证均验证安全性与模型性能优势
深度学习模型因能表征分层特征而被广泛应用于多个领域，其性能高度依赖训练数据与过程。因此，保护训练过程和算法对隐私保护至关重要。尽管差分隐私（DP）作为强大的密码学工具已在深度学习训练中取得良好效果，但现有方案仍存在模型效用下降的问题，即要么引入过高噪声，要么不可避免地损害原始梯度。为解决上述问题，本文提出一种更鲁棒且可证明安全的差分隐私训练方法GReDP。具体而言，我们在频域中计算模型梯度，并采用新方法降低噪声水平。与以往工作不同，GReDP仅需DPSGD[1]一半的噪声尺度即可完整保留所有梯度信息。我们从理论和实验两方面详细分析了该方法。实验结果表明，GReDP在所有模型和训练设置下均优于基线方法。
原文摘要 · Abstract (English)
Deep learning models have been extensively adopted in various regions due to their ability to represent hierarchical features, which highly rely on the training set and procedures. Thus, protecting the training process and deep learning algorithms is paramount in privacy preservation. Although Differential Privacy (DP) as a powerful cryptographic primitive has achieved satisfying results in deep learning training, the existing schemes still fall short in preserving model utility, i.e., they either invoke a high noise scale or inevitably harm the original gradients. To address the above issues, in this paper, we present a more robust and provably secure approach for differentially private training called GReDP. Specifically, we compute the model gradients in the frequency domain and adopt a new approach to reduce the noise level. Unlike previous work, our GReDP only requires half of the noise scale compared to DPSGD [1] while keeping all the gradient information intact. We present a detailed analysis of our method both theoretically and empirically. The experimental results show that our GReDP works consistently better than the baselines on all models and training settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。