用随机投影加噪保护隐私,生成模型训练更灵活高效
Privacy without Noisy Gradients: Slicing Mechanism for Generative Model Training
- 通过低维投影加噪实现强隐私保护,避免梯度噪声
- 新提出平滑切片f散度,支持非对抗训练且收敛稳定
- 可自由调整网络结构与训练轮数,无额外隐私损耗
使用差分隐私(DP)训练生成模型通常依赖于在梯度更新中注入噪声或修改判别器训练过程,导致超参数调优困难且难以收敛。本文提出一种切片隐私机制,对私有数据的随机低维投影注入噪声,并提供强有力的隐私保证。这些带噪投影用于生成模型训练。为支持该方法优化生成模型,我们引入平滑切片f散度,并证明其具有统计一致性。此外,我们设计了一种基于核的方法估计该散度,无需对抗训练。大量数值实验表明,该方法生成的合成数据质量优于基线。更重要的是,由于不依赖噪声梯度,数据科学家可自由调整生成器架构、超参数,运行任意轮次训练甚至重启优化,均不增加额外隐私开销。
原文摘要 · Abstract (English)
Training generative models with differential privacy (DP) typically involves injecting noise into gradient updates or adapting the discriminator's training procedure. As a result, such approaches often struggle with hyper-parameter tuning and convergence. We consider the slicing privacy mechanism that injects noise into random low-dimensional projections of the private data, and provide strong privacy guarantees for it. These noisy projections are used for training generative models. To enable optimizing generative models using this DP approach, we introduce the smoothed-sliced $f$-divergence and show it enjoys statistical consistency. Moreover, we present a kernel-based estimator for this divergence, circumventing the need for adversarial training. Extensive numerical experiments demonstrate that our approach can generate synthetic data of higher quality compared with baselines. Beyond performance improvement, our method, by sidestepping the need for noisy gradients, offers data scientists the flexibility to adjust generator architecture and hyper-parameters, run the optimization over any number of epochs, and even restart the optimization process -- all without incurring additional privacy costs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。