用模拟器替代大模型生成隐私保护合成数据,效果更好且更高效
Differentially Private Synthetic Data via APIs 3: Using Simulators Instead of Foundation Model
- 将图像生成模拟器接入私密数据合成框架,无需神经网络
- 合成图像下游分类准确率提升3倍,FID降低80%
- 适合想用高效模拟器做隐私数据生成的研究者和工程师
差分隐私(DP)合成数据能够在保持强隐私保障的同时,高度还原原始数据,已成为释放私有数据价值的关键工具。近期,私有演化(PE)成为生成DP合成数据的有前景方法。与基于训练的方法不同,PE仅需访问基础模型的推理API,可直接利用最先进的模型能力。然而,特定私有数据领域未必存在合适的预训练模型。本文发现,PE框架足够通用,可支持除基础模型外的各类API。特别是我们证明,许多不依赖神经网络的先进数据合成器——如基于计算机图形学的图像生成器(称为模拟器)——可有效集成到PE中。这一发现显著拓展了PE的应用范围,释放了强大模拟器在DP数据合成中的潜力。我们在图像合成场景下探索该方法,命名为Sim-PE。在四个不同模拟器上,Sim-PE表现优异:相比原PE,下游分类准确率最高提升3倍,FID降低最多达80%,且效率更高。我们还展示了模拟器与基础模型可在PE框架内协同使用,进一步提升性能。代码已开源至Private Evolution Python库:https://github.com/microsoft/DPSDA。
原文摘要 · Abstract (English)
Differentially private (DP) synthetic data, which closely resembles the original private data while maintaining strong privacy guarantees, has become a key tool for unlocking the value of private data without compromising privacy. Recently, Private Evolution (PE) has emerged as a promising method for generating DP synthetic data. Unlike other training-based approaches, PE only requires access to inference APIs from foundation models, enabling it to harness the power of state-of-the-art (SoTA) models. However, a suitable foundation model for a specific private data domain is not always available. In this paper, we discover that the PE framework is sufficiently general to allow APIs beyond foundation models. In particular, we demonstrate that many SoTA data synthesizers that do not rely on neural networks--such as computer graphics-based image generators, which we refer to as simulators--can be effectively integrated into PE. This insight significantly broadens PE's applicability and unlocks the potential of powerful simulators for DP data synthesis. We explore this approach, named Sim-PE, in the context of image synthesis. Across four diverse simulators, Sim-PE performs well, improving the downstream classification accuracy of PE by up to 3x, reducing FID by up to 80%, and offering much greater efficiency. We also show that simulators and foundation models can be easily leveraged together within PE to achieve further improvements. The code is open-sourced in the Private Evolution Python library: https://github.com/microsoft/DPSDA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。