用大模型生成带差分隐私的合成数据,保护隐私同时保持数据可用
SafeSynthDP: Leveraging Large Language Models for Privacy-Preserving Synthetic Data Generation Using Differential Privacy
- 在大模型生成数据时注入拉普拉斯/高斯噪声实现差分隐私
- 合成数据训练的模型性能接近真实数据训练结果,隐私泄露风险低
- 适合关注数据合规与隐私保护的研究者和从业者
机器学习模型常依赖包含敏感信息的训练数据,引发严重隐私担忧。欧盟通用数据保护条例(GDPR)和加州消费者隐私法(CCPA)等法规推动了在保障数据效用的同时保护隐私的策略发展。本文研究大语言模型(LLMs)结合差分隐私(DP)机制生成合成数据的能力,实现在不暴露原始敏感信息的前提下支持数据驱动研究与模型训练。方法上,将基于拉普拉斯和高斯分布的差分隐私噪声注入数据生成过程;通过对比在合成数据与原始数据上训练的机器学习模型性能,评估数据效用;并通过成员推断攻击等威胁测试生成数据的隐私韧性。实验表明,将差分隐私融入大模型合成数据生成,可在隐私保护与数据效用之间取得可行平衡。本研究为大模型的隐私保护能力提供了基础方法与洞察,推动符合法规的高效机器学习应用。
原文摘要 · Abstract (English)
Machine learning (ML) models frequently rely on training data that may include sensitive or personal information, raising substantial privacy concerns. Legislative frameworks such as the General Data Protection Regulation (GDPR) and the California Consumer Privacy Act (CCPA) have necessitated the development of strategies that preserve privacy while maintaining the utility of data. In this paper, we investigate the capability of Large Language Models (LLMs) to generate synthetic datasets integrated with Differential Privacy (DP) mechanisms, thereby enabling data-driven research and model training without direct exposure of sensitive information. Our approach incorporates DP-based noise injection methods, including Laplace and Gaussian distributions, into the data generation process. We then evaluate the utility of these DP-enhanced synthetic datasets by comparing the performance of ML models trained on them against models trained on the original data. To substantiate privacy guarantees, we assess the resilience of the generated synthetic data to membership inference attacks and related threats. The experimental results demonstrate that integrating DP within LLM-driven synthetic data generation offers a viable balance between privacy protection and data utility. This study provides a foundational methodology and insight into the privacy-preserving capabilities of LLMs, paving the way for compliant and effective ML research and applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。